Aug 03 23:27:48.695055 ip-10-0-143-183 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Aug 03 23:27:48.695067 ip-10-0-143-183 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Aug 03 23:27:48.695077 ip-10-0-143-183 systemd[1]: kubelet.service: Failed with result 'resources'.
Aug 03 23:27:48.695402 ip-10-0-143-183 systemd[1]: Failed to start Kubernetes Kubelet.
Aug 03 23:27:58.793859 ip-10-0-143-183 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Aug 03 23:27:58.793877 ip-10-0-143-183 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 98ecc7704ea740f0b2afe233945d85e8 --
Aug 03 23:29:52.195991 ip-10-0-143-183 systemd[1]: Starting Kubernetes Kubelet...
Aug 03 23:29:52.548257 ip-10-0-143-183 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 03 23:29:52.548257 ip-10-0-143-183 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 03 23:29:52.548257 ip-10-0-143-183 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 03 23:29:52.548257 ip-10-0-143-183 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 03 23:29:52.548257 ip-10-0-143-183 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 03 23:29:52.549214 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.549111    2553 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 03 23:29:52.552968 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.552954    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 23:29:52.552968 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.552968    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.552972    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.552977    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.552981    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.552984    2553 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.552987    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.552990    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.552993    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.552996    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.552998    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553001    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553003    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553007    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553010    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553013    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553016    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553018    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553028    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553031    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 23:29:52.553029 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553034    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553036    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553039    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553042    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553045    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553047    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553050    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553053    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553055    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553057    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553060    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553063    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553065    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553067    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553070    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553072    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553075    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553077    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553080    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 23:29:52.553481 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553085    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553088    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553091    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553094    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553096    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553099    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553102    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553105    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553107    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553110    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553112    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553114    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553117    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553119    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553122    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553124    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553127    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553129    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553132    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553134    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 23:29:52.554001 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553137    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553139    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553142    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553144    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553147    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553149    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553151    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553154    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553156    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553158    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553161    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553163    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553166    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553169    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553171    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553174    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553176    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553179    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553181    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553184    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 23:29:52.554473 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553187    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553189    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553192    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553194    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553197    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553199    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553202    2553 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553204    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553207    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553209    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553212    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553216    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553222    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553226    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553231    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553235    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553239    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553243    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553247    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 23:29:52.554982 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.553250    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 23:29:52.556629 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556617    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 23:29:52.556629 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556630    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556633    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556638    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556642    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556645    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556648    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556651    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556655    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556659    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556662    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556665    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556668    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556671    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556674    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556676    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556679    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556681    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556684    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556686    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 23:29:52.556687 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556689    2553 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556692    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556695    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556697    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556700    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556715    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556718    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556721    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556723    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556726    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556729    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556731    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556734    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556737    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556740    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556743    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556745    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556748    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556750    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556753    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 23:29:52.557164 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556755    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556759    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556763    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556766    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556768    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556771    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556774    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556776    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556780    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556782    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556785    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556787    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556789    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556792    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556795    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556798    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556800    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556803    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556805    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 23:29:52.557679 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556808    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556810    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556813    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556815    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556818    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556820    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556823    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556825    2553 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556828    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556830    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556833    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556835    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556838    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556840    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556843    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556845    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556847    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556850    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556852    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 23:29:52.558152 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556855    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556857    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556861    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556863    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556867    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556869    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556872    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556874    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556877    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556880    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556882    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556885    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556887    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556890    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556892    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556895    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556897    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556900    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556902    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556905    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 23:29:52.558746 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.556907    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557275    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557280    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557283    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557286    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557290    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557292    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557295    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557298    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557300    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557303    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557305    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557308    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557310    2553 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557313    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557315    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557318    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557321    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557323    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557326    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 23:29:52.559235 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557330    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557332    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557335    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557337    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557340    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557342    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557345    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557347    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557349    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557352    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557354    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557358    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557361    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557364    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557367    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557370    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557373    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557375    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557378    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557381    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 23:29:52.559735 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557384    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557387    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557389    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557392    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557394    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557397    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557400    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557403    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557405    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557408    2553 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557410    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557413    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557416    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557418    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557421    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557423    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557426    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557428    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557431    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 23:29:52.560214 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557433    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557437    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557440    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557442    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557445    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557447    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557449    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557452    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557454    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557457    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557460    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557462    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557465    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557468    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557470    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557472    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557475    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557478    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557482    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 23:29:52.560693 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557485    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557488    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557491    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557493    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557495    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557498    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557500    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557503    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557505    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557508    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557510    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557513    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557515    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557518    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557520    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557523    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557525    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557528    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557530    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557532    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 23:29:52.561159 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557535    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.557537    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557603    2553 flags.go:64] FLAG: --address="0.0.0.0"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557610    2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557616    2553 flags.go:64] FLAG: --anonymous-auth="true"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557620    2553 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557624    2553 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557627    2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557630    2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557634    2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557637    2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557640    2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557646    2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557649    2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557652    2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557655    2553 flags.go:64] FLAG: --cgroup-root=""
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557657    2553 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557660    2553 flags.go:64] FLAG: --client-ca-file=""
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557663    2553 flags.go:64] FLAG: --cloud-provider="external"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557666    2553 flags.go:64] FLAG: --cluster-dns="[]"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557669    2553 flags.go:64] FLAG: --cluster-domain=""
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557672    2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557675    2553 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557678    2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 03 23:29:52.561643 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557681    2553 flags.go:64] FLAG: --container-log-max-files="5"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557685    2553 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557688    2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557691    2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557694    2553 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557697    2553 flags.go:64] FLAG: --contention-profiling="false"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557700    2553 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557719    2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557723    2553 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557726    2553 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557730    2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557733    2553 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557736    2553 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557739    2553 flags.go:64] FLAG: --enable-load-reader="false"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557742    2553 flags.go:64] FLAG: --enable-server="true"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557745    2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557750    2553 flags.go:64] FLAG: --event-burst="100"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557753    2553 flags.go:64] FLAG: --event-qps="50"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557755    2553 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557758    2553 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557762    2553 flags.go:64] FLAG: --eviction-hard=""
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557765    2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557768    2553 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557771    2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557773    2553 flags.go:64] FLAG: --eviction-soft=""
Aug 03 23:29:52.562253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557776    2553 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557779    2553 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557781    2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557784    2553 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557787    2553 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557790    2553 flags.go:64] FLAG: --fail-swap-on="true"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557792    2553 flags.go:64] FLAG: --feature-gates=""
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557796    2553 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557798    2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557801    2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557804    2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557807    2553 flags.go:64] FLAG: --healthz-port="10248"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557810    2553 flags.go:64] FLAG: --help="false"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557812    2553 flags.go:64] FLAG: --hostname-override="ip-10-0-143-183.ec2.internal"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557815    2553 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557818    2553 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557821    2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557824    2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557828    2553 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557831    2553 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557834    2553 flags.go:64] FLAG: --image-service-endpoint=""
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557837    2553 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557840    2553 flags.go:64] FLAG: --kube-api-burst="100"
Aug 03 23:29:52.562867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557843    2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557846    2553 flags.go:64] FLAG: --kube-api-qps="50"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557849    2553 flags.go:64] FLAG: --kube-reserved=""
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557852    2553 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557854    2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557857    2553 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557860    2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557863    2553 flags.go:64] FLAG: --lock-file=""
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557865    2553 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557868    2553 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557870    2553 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557875    2553 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557878    2553 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557880    2553 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557883    2553 flags.go:64] FLAG: --logging-format="text"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557885    2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557889    2553 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557891    2553 flags.go:64] FLAG: --manifest-url=""
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557894    2553 flags.go:64] FLAG: --manifest-url-header=""
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557897    2553 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557900    2553 flags.go:64] FLAG: --max-open-files="1000000"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557904    2553 flags.go:64] FLAG: --max-pods="110"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557907    2553 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557909    2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557912    2553 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 03 23:29:52.563565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557915    2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557918    2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557922    2553 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557925    2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557932    2553 flags.go:64] FLAG: --node-status-max-images="50"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557934    2553 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557937    2553 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557940    2553 flags.go:64] FLAG: --pod-cidr=""
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557944    2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557949    2553 flags.go:64] FLAG: --pod-manifest-path=""
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557952    2553 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557955    2553 flags.go:64] FLAG: --pods-per-core="0"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557957    2553 flags.go:64] FLAG: --port="10250"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557960    2553 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557963    2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02b73ef45dd89cf5f"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557966    2553 flags.go:64] FLAG: --qos-reserved=""
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557969    2553 flags.go:64] FLAG: --read-only-port="10255"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557971    2553 flags.go:64] FLAG: --register-node="true"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557974    2553 flags.go:64] FLAG: --register-with-taints=""
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557977    2553 flags.go:64] FLAG: --registry-burst="10"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557980    2553 flags.go:64] FLAG: --registry-qps="5"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557982    2553 flags.go:64] FLAG: --reserved-cpus=""
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557985    2553 flags.go:64] FLAG: --reserved-memory=""
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557988    2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557991    2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 03 23:29:52.564635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557994    2553 flags.go:64] FLAG: --rotate-certificates="false"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557997    2553 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.557999    2553 flags.go:64] FLAG: --runonce="false"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558002    2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558005    2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558007    2553 flags.go:64] FLAG: --seccomp-default="false"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558010    2553 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558012    2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558015    2553 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558019    2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558022    2553 flags.go:64] FLAG: --storage-driver-password="root"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558024    2553 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558027    2553 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558030    2553 flags.go:64] FLAG: --storage-driver-user="root"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558033    2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558035    2553 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558038    2553 flags.go:64] FLAG: --system-cgroups=""
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558041    2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558046    2553 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558049    2553 flags.go:64] FLAG: --tls-cert-file=""
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558051    2553 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558055    2553 flags.go:64] FLAG: --tls-min-version=""
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558058    2553 flags.go:64] FLAG: --tls-private-key-file=""
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558060    2553 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558063    2553 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 03 23:29:52.565692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558066    2553 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558068    2553 flags.go:64] FLAG: --v="2"
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558072    2553 flags.go:64] FLAG: --version="false"
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558075    2553 flags.go:64] FLAG: --vmodule=""
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558079    2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558082    2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558190    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558193    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558196    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558198    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558201    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558204    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558206    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558208    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558212    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558215    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558220    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558222    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558225    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558229    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558233    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 23:29:52.566883 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558236    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558239    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558242    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558245    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558248    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558251    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558253    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558256    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558259    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558262    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558265    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558268    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558271    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558274    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558277    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558280    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558282    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558284    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558287    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 23:29:52.567526 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558289    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558292    2553 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558294    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558297    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558299    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558302    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558304    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558307    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558311    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558313    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558315    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558318    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558321    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558324    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558327    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558330    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558332    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558339    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558342    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558344    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 23:29:52.568321 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558347    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558349    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558351    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558354    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558356    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558358    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558361    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558363    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558366    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558368    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558371    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558373    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558376    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558378    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558380    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558383    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558385    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558388    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558390    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 23:29:52.569085 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558393    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558396    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558399    2553 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558401    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558403    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558407    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558410    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558412    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558415    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558417    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558419    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558423    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558425    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558428    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558430    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558433    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558435    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558437    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558440    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558443    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 23:29:52.569683 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558445    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558448    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558450    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558452    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558455    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.558457    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.558462    2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.565553    2553 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.565570    2553 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.565592    2553 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.565605    2553 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565726    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565734    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565739    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565744    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 23:29:52.570272 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565749    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565753    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565758    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565762    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565766    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565771    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565775    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565780    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565784    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565788    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565792    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565796    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565801    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565808    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565813    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565818    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565822    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565827    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565832    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565836    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 23:29:52.570719 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565841    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565845    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565849    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565853    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565858    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565864    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565869    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565873    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565878    2553 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565882    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565887    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565891    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565895    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565899    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565903    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565907    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565911    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565915    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565919    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565925    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 23:29:52.571240 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565931    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565935    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565940    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565944    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565948    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565953    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565957    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565961    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565965    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565969    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565973    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565977    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565981    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565985    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565989    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565993    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.565998    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566002    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566007    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 23:29:52.571784 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566011    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566015    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566020    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566024    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566028    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566032    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566036    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566040    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566044    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566048    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566052    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566056    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566060    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566064    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566068    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566072    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566079    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566084    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566089    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 23:29:52.572258 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566093    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566098    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566103    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566108    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566113    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566117    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566122    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566126    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566131    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566135    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566141    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566145    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566150    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566154    2553 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566158    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566163    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 23:29:52.572933 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566167    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.566175    2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566351    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566359    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566364    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566369    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566374    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566378    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566382    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566386    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566390    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566397    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566403    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566408    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566413    2553 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 23:29:52.573584 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566417    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566421    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566426    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566431    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566435    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566439    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566443    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566447    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566452    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566458    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566462    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566466    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566471    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566476    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566480    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566484    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566488    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566493    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566498    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 23:29:52.574194 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566502    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566506    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566510    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566514    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566519    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566522    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566526    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566530    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566534    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566539    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566542    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566546    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566551    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566555    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566559    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566563    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566567    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566571    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566574    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 23:29:52.574843 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566579    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566583    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566587    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566591    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566595    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566599    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566603    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566607    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566611    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566615    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566620    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566625    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566629    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566633    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566637    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566650    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566654    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566659    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566662    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566666    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566670    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 23:29:52.575431 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566674    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566678    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566682    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566686    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566690    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566694    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566698    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566702    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566724    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566728    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566732    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566736    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566739    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566743    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566746    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566750    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566754    2553 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566757    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566761    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566765    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 23:29:52.576025 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566769    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566773    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566777    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566784    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566789    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566794    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:52.566798    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.566806    2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.566936    2553 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.569383    2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.570569    2553 server.go:1019] "Starting client certificate rotation"
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.570671    2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.570699    2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 03 23:29:52.576487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.574218    2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 03 23:29:52.577993 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.577978    2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 03 23:29:52.587618 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.587601    2553 log.go:25] "Validated CRI v1 runtime API"
Aug 03 23:29:52.590416 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.590403    2553 log.go:25] "Validated CRI v1 image API"
Aug 03 23:29:52.591240 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.591224    2553 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 03 23:29:52.595118 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.595094    2553 fs.go:135] Filesystem UUIDs: map[0aefdd2c-422d-4361-847b-0a36ccb33406:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c18f4aba-56d1-4f16-b098-6c70990e4aee:/dev/nvme0n1p4]
Aug 03 23:29:52.595178 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.595118    2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 03 23:29:52.597723 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.597690    2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 03 23:29:52.601038 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.600941    2553 manager.go:216] Machine: {Timestamp:2026-08-03 23:29:52.599648387 +0000 UTC m=+0.301355042 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3136171 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c16c209bc26bd950346dad7c7bbe5 SystemUUID:ec2c16c2-09bc-26bd-9503-46dad7c7bbe5 BootID:98ecc770-4ea7-40f0-b2af-e233945d85e8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:91:2b:e6:b6:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:91:2b:e6:b6:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:d4:3d:e7:c1:fd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 03 23:29:52.601038 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.601030    2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 03 23:29:52.601160 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.601098    2553 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 03 23:29:52.604006 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.603985    2553 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 03 23:29:52.604176 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.604010    2553 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-183.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 03 23:29:52.604729 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.604719    2553 topology_manager.go:138] "Creating topology manager with none policy"
Aug 03 23:29:52.604760 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.604732    2553 container_manager_linux.go:309] "Creating device plugin manager"
Aug 03 23:29:52.604760 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.604738    2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 03 23:29:52.605427 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.605418    2553 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 03 23:29:52.606165 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.606155    2553 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 03 23:29:52.609693 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.609682    2553 state_mem.go:36] "Initialized new in-memory state store"
Aug 03 23:29:52.609848 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.609838    2553 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 03 23:29:52.609884 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.609868    2553 kubelet.go:486] "Attempting to sync node with API server"
Aug 03 23:29:52.609884 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.609877    2553 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 03 23:29:52.609941 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.609895    2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 03 23:29:52.609941 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.609904    2553 kubelet.go:398] "Adding apiserver pod source"
Aug 03 23:29:52.609941 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.609912    2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 03 23:29:52.610256 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.610243    2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 03 23:29:52.610324 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.610262    2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 03 23:29:52.613199 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.613186    2553 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 03 23:29:52.616236 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.616038    2553 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 03 23:29:52.616296 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.616255    2553 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 03 23:29:52.617241 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617229    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 03 23:29:52.617282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617246    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 03 23:29:52.617282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617252    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 03 23:29:52.617282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617258    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 03 23:29:52.617282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617266    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 03 23:29:52.617282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617271    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 03 23:29:52.617282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617279    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 03 23:29:52.617282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617285    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 03 23:29:52.617453 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617292    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 03 23:29:52.617453 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617297    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 03 23:29:52.617453 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617307    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 03 23:29:52.617453 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617312    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 03 23:29:52.617901 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617890    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 03 23:29:52.617931 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.617904    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 03 23:29:52.619683 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.619666    2553 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-183.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 03 23:29:52.620049 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.620032    2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-183.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 03 23:29:52.620089 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.620066    2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 03 23:29:52.620977 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.620965    2553 server.go:1268] "Started kubelet"
Aug 03 23:29:52.621102 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.621043    2553 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 03 23:29:52.621166 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.621078    2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 03 23:29:52.621212 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.621124    2553 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 03 23:29:52.621281 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.621262    2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mfn6l"
Aug 03 23:29:52.621790 ip-10-0-143-183 systemd[1]: Started Kubernetes Kubelet.
Aug 03 23:29:52.622424 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.622330    2553 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 03 23:29:52.622842 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.622697    2553 server.go:310] "Adding debug handlers to kubelet server"
Aug 03 23:29:52.626556 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.626537    2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mfn6l"
Aug 03 23:29:52.626636 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.626574    2553 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 03 23:29:52.629463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.629447    2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 03 23:29:52.629549 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.629469    2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 03 23:29:52.629608 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.629544    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:52.630021 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630001    2553 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 03 23:29:52.630021 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630023    2553 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 03 23:29:52.630146 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630001    2553 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 03 23:29:52.630581 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630564    2553 reconstruct.go:97] "Volume reconstruction finished"
Aug 03 23:29:52.630581 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630579    2553 reconciler.go:29] "Reconciler: start to sync state"
Aug 03 23:29:52.630698 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630606    2553 factory.go:153] Registering CRI-O factory
Aug 03 23:29:52.630698 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630618    2553 factory.go:223] Registration of the crio container factory successfully
Aug 03 23:29:52.630698 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630667    2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 03 23:29:52.630698 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630674    2553 factory.go:55] Registering systemd factory
Aug 03 23:29:52.630698 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630682    2553 factory.go:223] Registration of the systemd container factory successfully
Aug 03 23:29:52.630698 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630701    2553 factory.go:103] Registering Raw factory
Aug 03 23:29:52.630698 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.630730    2553 manager.go:1196] Started watching for new ooms in manager
Aug 03 23:29:52.631355 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.631330    2553 manager.go:349] Starting recovery of all containers
Aug 03 23:29:52.635333 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.635309    2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 23:29:52.639237 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.639216    2553 manager.go:354] Recovery completed
Aug 03 23:29:52.639905 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.639886    2553 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-183.ec2.internal\" not found" node="ip-10-0-143-183.ec2.internal"
Aug 03 23:29:52.647005 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.646908    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 23:29:52.648923 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.648905    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 23:29:52.648997 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.648937    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 23:29:52.648997 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.648951    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientPID"
Aug 03 23:29:52.649408 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.649394    2553 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 03 23:29:52.649461 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.649408    2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 03 23:29:52.649461 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.649424    2553 state_mem.go:36] "Initialized new in-memory state store"
Aug 03 23:29:52.649538 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.649401    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 23:29:52.650102 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.650089    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 23:29:52.650151 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.650111    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 23:29:52.650151 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.650124    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientPID"
Aug 03 23:29:52.650357 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.650346    2553 policy_none.go:49] "None policy: Start"
Aug 03 23:29:52.650392 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.650362    2553 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 03 23:29:52.650392 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.650372    2553 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 03 23:29:52.651228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.651218    2553 policy_none.go:47] "Start"
Aug 03 23:29:52.692284 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.692271    2553 manager.go:337] "Starting Device Plugin manager"
Aug 03 23:29:52.692374 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.692303    2553 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 03 23:29:52.692374 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.692315    2553 server.go:87] "Starting device plugin registration server"
Aug 03 23:29:52.692555 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.692542    2553 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 03 23:29:52.692618 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.692569    2553 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 03 23:29:52.692735 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.692684    2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 03 23:29:52.692809 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.692795    2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 03 23:29:52.692868 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.692809    2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 03 23:29:52.693242 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.693223    2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 03 23:29:52.693341 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.693274    2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:52.756319 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.756299    2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 03 23:29:52.758266 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.757509    2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 03 23:29:52.758266 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.757539    2553 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 03 23:29:52.758266 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.757555    2553 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 03 23:29:52.758266 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.757581    2553 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 03 23:29:52.759207 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.759185    2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 23:29:52.793526 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.793504    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 23:29:52.794240 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.794226    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 23:29:52.794306 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.794254    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 23:29:52.794306 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.794264    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientPID"
Aug 03 23:29:52.794306 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.794284    2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-183.ec2.internal"
Aug 03 23:29:52.805027 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.804993    2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-183.ec2.internal"
Aug 03 23:29:52.805027 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.805011    2553 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-183.ec2.internal\": node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:52.818140 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.818119    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:52.858622 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.858603    2553 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-183.ec2.internal"]
Aug 03 23:29:52.858682 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.858668    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 23:29:52.859425 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.859409    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 23:29:52.859487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.859441    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 23:29:52.859487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.859457    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientPID"
Aug 03 23:29:52.861621 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.861609    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 23:29:52.861804 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.861788    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:52.861857 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.861826    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 23:29:52.862331 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.862310    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 23:29:52.862423 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.862334    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 23:29:52.862423 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.862345    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientPID"
Aug 03 23:29:52.862423 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.862317    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 23:29:52.862423 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.862383    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 23:29:52.862423 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.862412    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientPID"
Aug 03 23:29:52.864549 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.864533    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:52.864634 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.864571    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 23:29:52.865197 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.865181    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 23:29:52.865301 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.865209    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 23:29:52.865301 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.865224    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeHasSufficientPID"
Aug 03 23:29:52.891212 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.891193    2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-183.ec2.internal\" not found" node="ip-10-0-143-183.ec2.internal"
Aug 03 23:29:52.895560 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.895544    2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-183.ec2.internal\" not found" node="ip-10-0-143-183.ec2.internal"
Aug 03 23:29:52.918823 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:52.918799    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:52.932368 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.932348    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/99cb8e83d430af053d61a47269a3a9b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal\" (UID: \"99cb8e83d430af053d61a47269a3a9b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:52.932437 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.932374    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99cb8e83d430af053d61a47269a3a9b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal\" (UID: \"99cb8e83d430af053d61a47269a3a9b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:52.932437 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:52.932392    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43e10beb2c51db4c48da59eafdb1be9c-config\") pod \"kube-apiserver-proxy-ip-10-0-143-183.ec2.internal\" (UID: \"43e10beb2c51db4c48da59eafdb1be9c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:53.019087 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:53.019056    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:53.033379 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.033356    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/99cb8e83d430af053d61a47269a3a9b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal\" (UID: \"99cb8e83d430af053d61a47269a3a9b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:53.033435 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.033426    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99cb8e83d430af053d61a47269a3a9b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal\" (UID: \"99cb8e83d430af053d61a47269a3a9b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:53.033473 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.033448    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99cb8e83d430af053d61a47269a3a9b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal\" (UID: \"99cb8e83d430af053d61a47269a3a9b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:53.033506 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.033478    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/99cb8e83d430af053d61a47269a3a9b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal\" (UID: \"99cb8e83d430af053d61a47269a3a9b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:53.033506 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.033489    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43e10beb2c51db4c48da59eafdb1be9c-config\") pod \"kube-apiserver-proxy-ip-10-0-143-183.ec2.internal\" (UID: \"43e10beb2c51db4c48da59eafdb1be9c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:53.033565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.033515    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43e10beb2c51db4c48da59eafdb1be9c-config\") pod \"kube-apiserver-proxy-ip-10-0-143-183.ec2.internal\" (UID: \"43e10beb2c51db4c48da59eafdb1be9c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:53.119825 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:53.119768    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:53.193294 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.193275    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:53.198404 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.198165    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:53.220738 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:53.220700    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:53.321272 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:53.321234    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:53.421785 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:53.421722    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:53.522256 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:53.522222    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:53.570678 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.570655    2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 03 23:29:53.571081 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.570796    2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 03 23:29:53.571081 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.570818    2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 03 23:29:53.622741 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:53.622720    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:53.630860 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.630837    2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-02 23:24:52 +0000 UTC" deadline="2028-02-24 19:30:09.257632217 +0000 UTC"
Aug 03 23:29:53.630944 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.630861    2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13676h0m15.626775237s"
Aug 03 23:29:53.723476 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:53.723404    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:53.752490 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:53.752460    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cb8e83d430af053d61a47269a3a9b8.slice/crio-9b9f064ae455442680f5c6fd3e1632e508c1a685f27538c0ce2c324250178fb4 WatchSource:0}: Error finding container 9b9f064ae455442680f5c6fd3e1632e508c1a685f27538c0ce2c324250178fb4: Status 404 returned error can't find the container with id 9b9f064ae455442680f5c6fd3e1632e508c1a685f27538c0ce2c324250178fb4
Aug 03 23:29:53.753461 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:53.753429    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e10beb2c51db4c48da59eafdb1be9c.slice/crio-5fb460fe3545da966b8f3c9158094ed302946613742fba7ab22cce36d78dd36e WatchSource:0}: Error finding container 5fb460fe3545da966b8f3c9158094ed302946613742fba7ab22cce36d78dd36e: Status 404 returned error can't find the container with id 5fb460fe3545da966b8f3c9158094ed302946613742fba7ab22cce36d78dd36e
Aug 03 23:29:53.758788 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.758770    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 23:29:53.760642 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.760567    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal" event={"ID":"99cb8e83d430af053d61a47269a3a9b8","Type":"ContainerStarted","Data":"9b9f064ae455442680f5c6fd3e1632e508c1a685f27538c0ce2c324250178fb4"}
Aug 03 23:29:53.761610 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.761584    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-183.ec2.internal" event={"ID":"43e10beb2c51db4c48da59eafdb1be9c","Type":"ContainerStarted","Data":"5fb460fe3545da966b8f3c9158094ed302946613742fba7ab22cce36d78dd36e"}
Aug 03 23:29:53.823649 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:53.823628    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-183.ec2.internal\" not found"
Aug 03 23:29:53.825016 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.824999    2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 23:29:53.830301 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.830284    2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:53.842476 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.842455    2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 03 23:29:53.844083 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.844070    2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-183.ec2.internal"
Aug 03 23:29:53.852687 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.852670    2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 03 23:29:53.965864 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:53.965842    2553 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 23:29:54.386975 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.386942    2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 23:29:54.611705 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.611672    2553 apiserver.go:52] "Watching apiserver"
Aug 03 23:29:54.621277 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.621250    2553 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 03 23:29:54.622120 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.622096    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-pfdwg","kube-system/konnectivity-agent-cw2jt","kube-system/kube-apiserver-proxy-ip-10-0-143-183.ec2.internal","openshift-dns/node-resolver-qm6s6","openshift-multus/network-metrics-daemon-kld8m","openshift-network-operator/iptables-alerter-rhsgn","openshift-ovn-kubernetes/ovnkube-node-n8bl5","kube-system/global-pull-secret-syncer-w526z","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh","openshift-cluster-node-tuning-operator/tuned-42jl2","openshift-image-registry/node-ca-wkhxt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal","openshift-multus/multus-additional-cni-plugins-xcf2p","openshift-multus/multus-g24wc"]
Aug 03 23:29:54.625311 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.625276    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.628259 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.628239    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 03 23:29:54.628504 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.628489    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 03 23:29:54.628796 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.628763    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 03 23:29:54.628981 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.628966    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 03 23:29:54.629216 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.629196    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 03 23:29:54.629427 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.629413    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.629911 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.629881    2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 03 23:29:54.629995 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.629937    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nnkp8\""
Aug 03 23:29:54.629995 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.629974    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 03 23:29:54.631552 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.631533    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.631997 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.631978    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 03 23:29:54.632095 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.632046    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 03 23:29:54.632095 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.632077    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-52d26\""
Aug 03 23:29:54.632217 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.631989    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 03 23:29:54.632381 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.632352    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 03 23:29:54.632456 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.632391    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 03 23:29:54.634042 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.633824    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jv85z\""
Aug 03 23:29:54.634042 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.633882    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 03 23:29:54.634205 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.634189    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qm6s6"
Aug 03 23:29:54.637229 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.637170    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:29:54.637324 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:54.637227    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:29:54.637907 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.637883    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 03 23:29:54.638010 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.637990    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 03 23:29:54.638114 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.638082    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hhcnt\""
Aug 03 23:29:54.639571 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.639550    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rhsgn"
Aug 03 23:29:54.641849 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.641807    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-node-log\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.641849 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.641842    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-cni-dir\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642005 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.641884    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-socket-dir-parent\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642005 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.641913    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-run-multus-certs\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642005 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.641938    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce4267c2-aafd-422e-b7cd-a21565cebf5b-env-overrides\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642005 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.641962    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce4267c2-aafd-422e-b7cd-a21565cebf5b-ovn-node-metrics-cert\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642005 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.641985    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/418e085f-c3bf-46eb-9ce1-9751c038c924-cni-binary-copy\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642029    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-var-lib-kubelet\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642054    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-daemon-config\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642077    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-run-ovn-kubernetes\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642099    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-cni-netd\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642123    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-cnibin\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.642254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642158    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/39619f91-b9cc-4d59-bcac-b0da494614ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.642254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642175    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-cnibin\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642190    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-var-lib-cni-bin\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642205    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-conf-dir\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642232    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-slash\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642248    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-etc-openvswitch\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642264    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-run-ovn\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642299    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-log-socket\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642314    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-system-cni-dir\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642330    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-system-cni-dir\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642347    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-systemd-units\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642376    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce4267c2-aafd-422e-b7cd-a21565cebf5b-ovnkube-config\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642390    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce4267c2-aafd-422e-b7cd-a21565cebf5b-ovnkube-script-lib\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642405    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39619f91-b9cc-4d59-bcac-b0da494614ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642419    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642455    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-kubelet\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642470    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-run-openvswitch\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642484    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-cni-bin\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642499    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-os-release\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642515    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-run-k8s-cni-cncf-io\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642541    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-var-lib-cni-multus\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.642756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642566    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-hostroot\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642601    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-run-netns\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642625    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-var-lib-openvswitch\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642686    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642715    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcxd8\" (UniqueName: \"kubernetes.io/projected/ce4267c2-aafd-422e-b7cd-a21565cebf5b-kube-api-access-wcxd8\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642756    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq4jb\" (UniqueName: \"kubernetes.io/projected/39619f91-b9cc-4d59-bcac-b0da494614ee-kube-api-access-fq4jb\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642897    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5jrxl\""
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642909    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-os-release\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642940    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-run-netns\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642964    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-run-systemd\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.642989    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/39619f91-b9cc-4d59-bcac-b0da494614ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.643016    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-etc-kubernetes\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.643051    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9dgf\" (UniqueName: \"kubernetes.io/projected/418e085f-c3bf-46eb-9ce1-9751c038c924-kube-api-access-j9dgf\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.643138    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.643326    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 03 23:29:54.643654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.643458    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 03 23:29:54.644432 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.643713    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.645259 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.644971    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:29:54.645259 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:54.645030    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:29:54.648025 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.646123    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 03 23:29:54.648025 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.646400    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 03 23:29:54.648025 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.646416    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 03 23:29:54.648025 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.646632    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2fgvc\""
Aug 03 23:29:54.648393 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.648239    2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 03 23:29:54.652453 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.652183    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:29:54.655179 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.654868    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 03 23:29:54.655179 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.654882    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 03 23:29:54.655179 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.655108    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cwqtl\""
Aug 03 23:29:54.656092 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.655746    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.659983 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.659906    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 03 23:29:54.666190 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.666166    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r62jc\""
Aug 03 23:29:54.666360 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.666172    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 03 23:29:54.667757 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.667739    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkhxt"
Aug 03 23:29:54.668950 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.668632    2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mdtx4"
Aug 03 23:29:54.669973 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.669956    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 03 23:29:54.670796 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.670312    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 03 23:29:54.670796 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.670379    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:54.670796 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:54.670445    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:29:54.670796 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.670528    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 03 23:29:54.670796 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.670680    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hbdz7\""
Aug 03 23:29:54.676292 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.676224    2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mdtx4"
Aug 03 23:29:54.731025 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.730974    2553 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 03 23:29:54.743279 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743242    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-run-ovn\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.743381 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743284    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-sysconfig\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.743381 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743312    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-sysctl-d\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.743381 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743338    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-systemd\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.743381 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743362    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6qqc\" (UniqueName: \"kubernetes.io/projected/ced51f35-5c98-4126-a3b4-533675741b86-kube-api-access-c6qqc\") pod \"node-resolver-qm6s6\" (UID: \"ced51f35-5c98-4126-a3b4-533675741b86\") " pod="openshift-dns/node-resolver-qm6s6"
Aug 03 23:29:54.743568 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743389    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-systemd-units\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.743568 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743414    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce4267c2-aafd-422e-b7cd-a21565cebf5b-ovnkube-script-lib\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.743568 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743433    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39619f91-b9cc-4d59-bcac-b0da494614ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.743568 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743448    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.743568 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743465    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-run\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.743568 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743489    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-var-lib-kubelet\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.743568 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743509    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/57500f92-3fda-430a-bbc0-37e3d091d27f-kubelet-config\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:54.743568 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743536    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-kubelet\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.743568 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743558    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-run-openvswitch\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743579    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-run-k8s-cni-cncf-io\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743596    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-var-lib-cni-multus\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743611    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-hostroot\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743625    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-kubernetes\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743652    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743680    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq4jb\" (UniqueName: \"kubernetes.io/projected/39619f91-b9cc-4d59-bcac-b0da494614ee-kube-api-access-fq4jb\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743705    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-os-release\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743722    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-host\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743743    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-registration-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743778    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-run-systemd\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743820    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9dgf\" (UniqueName: \"kubernetes.io/projected/418e085f-c3bf-46eb-9ce1-9751c038c924-kube-api-access-j9dgf\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743839    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c-host-slash\") pod \"iptables-alerter-rhsgn\" (UID: \"8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c\") " pod="openshift-network-operator/iptables-alerter-rhsgn"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743871    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/79fe88bf-1fbb-4616-9dbb-3a3ccab24265-serviceca\") pod \"node-ca-wkhxt\" (UID: \"79fe88bf-1fbb-4616-9dbb-3a3ccab24265\") " pod="openshift-image-registry/node-ca-wkhxt"
Aug 03 23:29:54.743937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743898    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-node-log\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743957    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-cni-dir\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.743984    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-run-multus-certs\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744010    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br2lw\" (UniqueName: \"kubernetes.io/projected/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-kube-api-access-br2lw\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744035    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/57500f92-3fda-430a-bbc0-37e3d091d27f-dbus\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744086    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744113    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjktt\" (UniqueName: \"kubernetes.io/projected/5e420d03-b0de-4f89-936a-cdafff2847a9-kube-api-access-fjktt\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744138    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce4267c2-aafd-422e-b7cd-a21565cebf5b-env-overrides\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744163    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce4267c2-aafd-422e-b7cd-a21565cebf5b-ovn-node-metrics-cert\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744215    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/418e085f-c3bf-46eb-9ce1-9751c038c924-cni-binary-copy\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744238    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-cnibin\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744254    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-cnibin\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744270    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-conf-dir\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744287    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-slash\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744302    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-log-socket\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744318    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-system-cni-dir\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744334    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-system-cni-dir\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.744463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744368    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-modprobe-d\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744393    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744408    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-device-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744426    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/79fe88bf-1fbb-4616-9dbb-3a3ccab24265-host\") pod \"node-ca-wkhxt\" (UID: \"79fe88bf-1fbb-4616-9dbb-3a3ccab24265\") " pod="openshift-image-registry/node-ca-wkhxt"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744454    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce4267c2-aafd-422e-b7cd-a21565cebf5b-ovnkube-config\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744482    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-tuned\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744501    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-socket-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744527    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-cni-bin\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744555    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-os-release\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744580    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ced51f35-5c98-4126-a3b4-533675741b86-hosts-file\") pod \"node-resolver-qm6s6\" (UID: \"ced51f35-5c98-4126-a3b4-533675741b86\") " pod="openshift-dns/node-resolver-qm6s6"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744599    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-etc-kubernetes\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744626    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6a69b3b6-18d1-4f41-8529-ad97331b0aa6-konnectivity-ca\") pod \"konnectivity-agent-cw2jt\" (UID: \"6a69b3b6-18d1-4f41-8529-ad97331b0aa6\") " pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744652    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-run-netns\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744686    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-var-lib-openvswitch\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744712    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcxd8\" (UniqueName: \"kubernetes.io/projected/ce4267c2-aafd-422e-b7cd-a21565cebf5b-kube-api-access-wcxd8\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744737    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-run-netns\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744800    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-lib-modules\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.745134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744827    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c-iptables-alerter-script\") pod \"iptables-alerter-rhsgn\" (UID: \"8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c\") " pod="openshift-network-operator/iptables-alerter-rhsgn"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744850    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/39619f91-b9cc-4d59-bcac-b0da494614ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744871    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6a69b3b6-18d1-4f41-8529-ad97331b0aa6-agent-certs\") pod \"konnectivity-agent-cw2jt\" (UID: \"6a69b3b6-18d1-4f41-8529-ad97331b0aa6\") " pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744887    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6czr\" (UniqueName: \"kubernetes.io/projected/8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c-kube-api-access-z6czr\") pod \"iptables-alerter-rhsgn\" (UID: \"8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c\") " pod="openshift-network-operator/iptables-alerter-rhsgn"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744902    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744919    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dswbq\" (UniqueName: \"kubernetes.io/projected/79fe88bf-1fbb-4616-9dbb-3a3ccab24265-kube-api-access-dswbq\") pod \"node-ca-wkhxt\" (UID: \"79fe88bf-1fbb-4616-9dbb-3a3ccab24265\") " pod="openshift-image-registry/node-ca-wkhxt"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744938    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-socket-dir-parent\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744966    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-var-lib-kubelet\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.744993    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-daemon-config\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745015    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfr8d\" (UniqueName: \"kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d\") pod \"network-check-target-pfdwg\" (UID: \"971b4204-55a9-4942-8e3c-c202fa0b93e2\") " pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745032    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-sys\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745047    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-tmp\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745062    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-sys-fs\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745085    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-run-ovn-kubernetes\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745108    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-cni-netd\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745139    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/39619f91-b9cc-4d59-bcac-b0da494614ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.745741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745165    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-var-lib-cni-bin\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.746477 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745190    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-sysctl-conf\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.746477 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745211    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:54.746477 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745228    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ced51f35-5c98-4126-a3b4-533675741b86-tmp-dir\") pod \"node-resolver-qm6s6\" (UID: \"ced51f35-5c98-4126-a3b4-533675741b86\") " pod="openshift-dns/node-resolver-qm6s6"
Aug 03 23:29:54.746477 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745244    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hhb8\" (UniqueName: \"kubernetes.io/projected/3e167335-fef0-4d29-bfb7-8d4271a429e6-kube-api-access-5hhb8\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.746477 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745267    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-etc-openvswitch\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.746477 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745358    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-etc-openvswitch\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.746477 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745646    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-run-ovn\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.746477 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.745708    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-systemd-units\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.746914    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39619f91-b9cc-4d59-bcac-b0da494614ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.746995    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747059    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-kubelet\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747123    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-run-openvswitch\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747143    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-run-netns\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747168    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-run-k8s-cni-cncf-io\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747194    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-var-lib-openvswitch\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747436    2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747542    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce4267c2-aafd-422e-b7cd-a21565cebf5b-env-overrides\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747600    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-var-lib-cni-multus\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747620    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-run-netns\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747637    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-hostroot\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747682    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747870    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-socket-dir-parent\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747917    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-var-lib-kubelet\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.747995    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-os-release\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748051    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-run-systemd\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.748859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748084    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/39619f91-b9cc-4d59-bcac-b0da494614ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748133    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-run-ovn-kubernetes\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748171    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-cni-netd\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748285    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-node-log\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748372    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-run-multus-certs\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748420    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-host-var-lib-cni-bin\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748429    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-daemon-config\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748432    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-cni-dir\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748478    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-cni-bin\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748481    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-system-cni-dir\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748498    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-os-release\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748574    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-multus-conf-dir\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748580    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/39619f91-b9cc-4d59-bcac-b0da494614ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748589    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce4267c2-aafd-422e-b7cd-a21565cebf5b-ovnkube-script-lib\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748619    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-cnibin\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748644    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-etc-kubernetes\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748694    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-log-socket\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748731    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce4267c2-aafd-422e-b7cd-a21565cebf5b-host-slash\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.749846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748752    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39619f91-b9cc-4d59-bcac-b0da494614ee-system-cni-dir\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.750702 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748767    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/418e085f-c3bf-46eb-9ce1-9751c038c924-cnibin\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.750702 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.748997    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce4267c2-aafd-422e-b7cd-a21565cebf5b-ovnkube-config\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.750702 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.749022    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/418e085f-c3bf-46eb-9ce1-9751c038c924-cni-binary-copy\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.753649 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.753627    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce4267c2-aafd-422e-b7cd-a21565cebf5b-ovn-node-metrics-cert\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.759991 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.759903    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9dgf\" (UniqueName: \"kubernetes.io/projected/418e085f-c3bf-46eb-9ce1-9751c038c924-kube-api-access-j9dgf\") pod \"multus-g24wc\" (UID: \"418e085f-c3bf-46eb-9ce1-9751c038c924\") " pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.759991 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.759938    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcxd8\" (UniqueName: \"kubernetes.io/projected/ce4267c2-aafd-422e-b7cd-a21565cebf5b-kube-api-access-wcxd8\") pod \"ovnkube-node-n8bl5\" (UID: \"ce4267c2-aafd-422e-b7cd-a21565cebf5b\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.761592 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.761569    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq4jb\" (UniqueName: \"kubernetes.io/projected/39619f91-b9cc-4d59-bcac-b0da494614ee-kube-api-access-fq4jb\") pod \"multus-additional-cni-plugins-xcf2p\" (UID: \"39619f91-b9cc-4d59-bcac-b0da494614ee\") " pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.846506 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846466    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:29:54.846670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846519    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjktt\" (UniqueName: \"kubernetes.io/projected/5e420d03-b0de-4f89-936a-cdafff2847a9-kube-api-access-fjktt\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:29:54.846670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846550    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-modprobe-d\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.846670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846565    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.846670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846587    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-device-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.846670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846604    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/79fe88bf-1fbb-4616-9dbb-3a3ccab24265-host\") pod \"node-ca-wkhxt\" (UID: \"79fe88bf-1fbb-4616-9dbb-3a3ccab24265\") " pod="openshift-image-registry/node-ca-wkhxt"
Aug 03 23:29:54.846670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846620    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-tuned\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.846670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846634    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-socket-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.846670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846654    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ced51f35-5c98-4126-a3b4-533675741b86-hosts-file\") pod \"node-resolver-qm6s6\" (UID: \"ced51f35-5c98-4126-a3b4-533675741b86\") " pod="openshift-dns/node-resolver-qm6s6"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846679    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6a69b3b6-18d1-4f41-8529-ad97331b0aa6-konnectivity-ca\") pod \"konnectivity-agent-cw2jt\" (UID: \"6a69b3b6-18d1-4f41-8529-ad97331b0aa6\") " pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846700    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-lib-modules\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846740    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c-iptables-alerter-script\") pod \"iptables-alerter-rhsgn\" (UID: \"8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c\") " pod="openshift-network-operator/iptables-alerter-rhsgn"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846764    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6a69b3b6-18d1-4f41-8529-ad97331b0aa6-agent-certs\") pod \"konnectivity-agent-cw2jt\" (UID: \"6a69b3b6-18d1-4f41-8529-ad97331b0aa6\") " pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846788    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6czr\" (UniqueName: \"kubernetes.io/projected/8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c-kube-api-access-z6czr\") pod \"iptables-alerter-rhsgn\" (UID: \"8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c\") " pod="openshift-network-operator/iptables-alerter-rhsgn"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846813    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846836    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dswbq\" (UniqueName: \"kubernetes.io/projected/79fe88bf-1fbb-4616-9dbb-3a3ccab24265-kube-api-access-dswbq\") pod \"node-ca-wkhxt\" (UID: \"79fe88bf-1fbb-4616-9dbb-3a3ccab24265\") " pod="openshift-image-registry/node-ca-wkhxt"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846864    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfr8d\" (UniqueName: \"kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d\") pod \"network-check-target-pfdwg\" (UID: \"971b4204-55a9-4942-8e3c-c202fa0b93e2\") " pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846886    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-sys\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846907    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-tmp\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846929    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-sys-fs\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846956    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-sysctl-conf\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.846979    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847001    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ced51f35-5c98-4126-a3b4-533675741b86-tmp-dir\") pod \"node-resolver-qm6s6\" (UID: \"ced51f35-5c98-4126-a3b4-533675741b86\") " pod="openshift-dns/node-resolver-qm6s6"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847024    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hhb8\" (UniqueName: \"kubernetes.io/projected/3e167335-fef0-4d29-bfb7-8d4271a429e6-kube-api-access-5hhb8\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.847052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847049    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-sysconfig\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847072    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-sysctl-d\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847097    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-systemd\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847126    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6qqc\" (UniqueName: \"kubernetes.io/projected/ced51f35-5c98-4126-a3b4-533675741b86-kube-api-access-c6qqc\") pod \"node-resolver-qm6s6\" (UID: \"ced51f35-5c98-4126-a3b4-533675741b86\") " pod="openshift-dns/node-resolver-qm6s6"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847153    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-run\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847175    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-var-lib-kubelet\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847199    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/57500f92-3fda-430a-bbc0-37e3d091d27f-kubelet-config\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847229    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-kubernetes\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847262    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-host\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847285    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-registration-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847313    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c-host-slash\") pod \"iptables-alerter-rhsgn\" (UID: \"8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c\") " pod="openshift-network-operator/iptables-alerter-rhsgn"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847339    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/79fe88bf-1fbb-4616-9dbb-3a3ccab24265-serviceca\") pod \"node-ca-wkhxt\" (UID: \"79fe88bf-1fbb-4616-9dbb-3a3ccab24265\") " pod="openshift-image-registry/node-ca-wkhxt"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847366    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br2lw\" (UniqueName: \"kubernetes.io/projected/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-kube-api-access-br2lw\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847391    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/57500f92-3fda-430a-bbc0-37e3d091d27f-dbus\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.847500    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/57500f92-3fda-430a-bbc0-37e3d091d27f-dbus\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:54.847615    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:29:54.847792 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:54.847682    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs podName:5e420d03-b0de-4f89-936a-cdafff2847a9 nodeName:}" failed. No retries permitted until 2026-08-03 23:29:55.34765662 +0000 UTC m=+3.049363276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs") pod "network-metrics-daemon-kld8m" (UID: "5e420d03-b0de-4f89-936a-cdafff2847a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:29:54.848365 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848007    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-modprobe-d\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.848365 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848068    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.848365 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848118    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-device-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.848365 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848157    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/79fe88bf-1fbb-4616-9dbb-3a3ccab24265-host\") pod \"node-ca-wkhxt\" (UID: \"79fe88bf-1fbb-4616-9dbb-3a3ccab24265\") " pod="openshift-image-registry/node-ca-wkhxt"
Aug 03 23:29:54.848518 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848483    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/57500f92-3fda-430a-bbc0-37e3d091d27f-kubelet-config\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:54.848569 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848542    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-systemd\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.848662 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848640    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-sysconfig\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.848741 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848667    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ced51f35-5c98-4126-a3b4-533675741b86-tmp-dir\") pod \"node-resolver-qm6s6\" (UID: \"ced51f35-5c98-4126-a3b4-533675741b86\") " pod="openshift-dns/node-resolver-qm6s6"
Aug 03 23:29:54.848798 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848751    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ced51f35-5c98-4126-a3b4-533675741b86-hosts-file\") pod \"node-resolver-qm6s6\" (UID: \"ced51f35-5c98-4126-a3b4-533675741b86\") " pod="openshift-dns/node-resolver-qm6s6"
Aug 03 23:29:54.848798 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848755    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-var-lib-kubelet\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.848962 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848803    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c-host-slash\") pod \"iptables-alerter-rhsgn\" (UID: \"8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c\") " pod="openshift-network-operator/iptables-alerter-rhsgn"
Aug 03 23:29:54.848962 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848840    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-host\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.848962 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848846    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-socket-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.848962 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848851    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-registration-dir\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.848962 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848923    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-sys-fs\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.849195 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.848918    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-sysctl-d\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.849195 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.849007    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-sysctl-conf\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.849195 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.849033    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-sys\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.849195 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.849063    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-run\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.849195 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:54.849093    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 23:29:54.849195 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:54.849126    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret podName:57500f92-3fda-430a-bbc0-37e3d091d27f nodeName:}" failed. No retries permitted until 2026-08-03 23:29:55.349115299 +0000 UTC m=+3.050821940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret") pod "global-pull-secret-syncer-w526z" (UID: "57500f92-3fda-430a-bbc0-37e3d091d27f") : object "kube-system"/"original-pull-secret" not registered
Aug 03 23:29:54.849195 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.849126    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-kubernetes\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.849514 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.849266    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e167335-fef0-4d29-bfb7-8d4271a429e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.849514 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.849316    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-lib-modules\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.849580 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.849532    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6a69b3b6-18d1-4f41-8529-ad97331b0aa6-konnectivity-ca\") pod \"konnectivity-agent-cw2jt\" (UID: \"6a69b3b6-18d1-4f41-8529-ad97331b0aa6\") " pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:29:54.849772 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.849753    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c-iptables-alerter-script\") pod \"iptables-alerter-rhsgn\" (UID: \"8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c\") " pod="openshift-network-operator/iptables-alerter-rhsgn"
Aug 03 23:29:54.849852 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.849795    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/79fe88bf-1fbb-4616-9dbb-3a3ccab24265-serviceca\") pod \"node-ca-wkhxt\" (UID: \"79fe88bf-1fbb-4616-9dbb-3a3ccab24265\") " pod="openshift-image-registry/node-ca-wkhxt"
Aug 03 23:29:54.853900 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.851079    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-etc-tuned\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.853900 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.852031    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-tmp\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.853900 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.853845    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6a69b3b6-18d1-4f41-8529-ad97331b0aa6-agent-certs\") pod \"konnectivity-agent-cw2jt\" (UID: \"6a69b3b6-18d1-4f41-8529-ad97331b0aa6\") " pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:29:54.858693 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:54.858669    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 23:29:54.858803 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:54.858697    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 23:29:54.858803 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:54.858728    2553 projected.go:196] Error preparing data for projected volume kube-api-access-sfr8d for pod openshift-network-diagnostics/network-check-target-pfdwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:29:54.858803 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:54.858781    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d podName:971b4204-55a9-4942-8e3c-c202fa0b93e2 nodeName:}" failed. No retries permitted until 2026-08-03 23:29:55.3587636 +0000 UTC m=+3.060470258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sfr8d" (UniqueName: "kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d") pod "network-check-target-pfdwg" (UID: "971b4204-55a9-4942-8e3c-c202fa0b93e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:29:54.860634 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.860612    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjktt\" (UniqueName: \"kubernetes.io/projected/5e420d03-b0de-4f89-936a-cdafff2847a9-kube-api-access-fjktt\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:29:54.861274 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.861248    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hhb8\" (UniqueName: \"kubernetes.io/projected/3e167335-fef0-4d29-bfb7-8d4271a429e6-kube-api-access-5hhb8\") pod \"aws-ebs-csi-driver-node-4hjwh\" (UID: \"3e167335-fef0-4d29-bfb7-8d4271a429e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:54.861453 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.861431    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6czr\" (UniqueName: \"kubernetes.io/projected/8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c-kube-api-access-z6czr\") pod \"iptables-alerter-rhsgn\" (UID: \"8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c\") " pod="openshift-network-operator/iptables-alerter-rhsgn"
Aug 03 23:29:54.861936 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.861895    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dswbq\" (UniqueName: \"kubernetes.io/projected/79fe88bf-1fbb-4616-9dbb-3a3ccab24265-kube-api-access-dswbq\") pod \"node-ca-wkhxt\" (UID: \"79fe88bf-1fbb-4616-9dbb-3a3ccab24265\") " pod="openshift-image-registry/node-ca-wkhxt"
Aug 03 23:29:54.862645 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.862626    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br2lw\" (UniqueName: \"kubernetes.io/projected/fcecf4ad-6c5e-4167-b617-5a9cc6015a24-kube-api-access-br2lw\") pod \"tuned-42jl2\" (UID: \"fcecf4ad-6c5e-4167-b617-5a9cc6015a24\") " pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:54.862760 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.862626    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6qqc\" (UniqueName: \"kubernetes.io/projected/ced51f35-5c98-4126-a3b4-533675741b86-kube-api-access-c6qqc\") pod \"node-resolver-qm6s6\" (UID: \"ced51f35-5c98-4126-a3b4-533675741b86\") " pod="openshift-dns/node-resolver-qm6s6"
Aug 03 23:29:54.943983 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.943898    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:29:54.951605 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.951582    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xcf2p"
Aug 03 23:29:54.971758 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.971731    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g24wc"
Aug 03 23:29:54.976327 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.976299    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qm6s6"
Aug 03 23:29:54.986776 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.986759    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rhsgn"
Aug 03 23:29:54.995449 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:54.995428    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh"
Aug 03 23:29:55.002121 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.002105    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:29:55.008886 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.008868    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-42jl2"
Aug 03 23:29:55.017630 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.017611    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkhxt"
Aug 03 23:29:55.058976 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.058951    2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 23:29:55.351511 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.351488    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:29:55.351617 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.351540    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:55.351654 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:55.351635    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 23:29:55.351654 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:55.351637    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:29:55.351744 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:55.351684    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret podName:57500f92-3fda-430a-bbc0-37e3d091d27f nodeName:}" failed. No retries permitted until 2026-08-03 23:29:56.3516713 +0000 UTC m=+4.053377942 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret") pod "global-pull-secret-syncer-w526z" (UID: "57500f92-3fda-430a-bbc0-37e3d091d27f") : object "kube-system"/"original-pull-secret" not registered
Aug 03 23:29:55.351744 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:55.351696    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs podName:5e420d03-b0de-4f89-936a-cdafff2847a9 nodeName:}" failed. No retries permitted until 2026-08-03 23:29:56.351690166 +0000 UTC m=+4.053396808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs") pod "network-metrics-daemon-kld8m" (UID: "5e420d03-b0de-4f89-936a-cdafff2847a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:29:55.452391 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.452353    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfr8d\" (UniqueName: \"kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d\") pod \"network-check-target-pfdwg\" (UID: \"971b4204-55a9-4942-8e3c-c202fa0b93e2\") " pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:29:55.452541 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:55.452526    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 23:29:55.452612 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:55.452546    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 23:29:55.452612 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:55.452556    2553 projected.go:196] Error preparing data for projected volume kube-api-access-sfr8d for pod openshift-network-diagnostics/network-check-target-pfdwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:29:55.452612 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:55.452610    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d podName:971b4204-55a9-4942-8e3c-c202fa0b93e2 nodeName:}" failed. No retries permitted until 2026-08-03 23:29:56.452592931 +0000 UTC m=+4.154299573 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfr8d" (UniqueName: "kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d") pod "network-check-target-pfdwg" (UID: "971b4204-55a9-4942-8e3c-c202fa0b93e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:29:55.681257 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.680099    2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-02 23:24:54 +0000 UTC" deadline="2028-04-07 13:28:22.627636214 +0000 UTC"
Aug 03 23:29:55.681257 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.680131    2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14701h58m26.947508494s"
Aug 03 23:29:55.702482 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:55.702449    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dfbd7e7_129f_4bf3_a9ea_3fb006b2a64c.slice/crio-50923380f8c7bd22064286655bb5d6d2364077e24c523b7330ac0b11a841ed2b WatchSource:0}: Error finding container 50923380f8c7bd22064286655bb5d6d2364077e24c523b7330ac0b11a841ed2b: Status 404 returned error can't find the container with id 50923380f8c7bd22064286655bb5d6d2364077e24c523b7330ac0b11a841ed2b
Aug 03 23:29:55.703936 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:55.703847    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced51f35_5c98_4126_a3b4_533675741b86.slice/crio-0266788ec9baa8835b5f1904097248160c7a09ca2820318bf4db443cb2577943 WatchSource:0}: Error finding container 0266788ec9baa8835b5f1904097248160c7a09ca2820318bf4db443cb2577943: Status 404 returned error can't find the container with id 0266788ec9baa8835b5f1904097248160c7a09ca2820318bf4db443cb2577943
Aug 03 23:29:55.704651 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:55.704622    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39619f91_b9cc_4d59_bcac_b0da494614ee.slice/crio-ab966f9127d9cf2c1c21785d6db73d4f9708813f889962d6d256a55975371926 WatchSource:0}: Error finding container ab966f9127d9cf2c1c21785d6db73d4f9708813f889962d6d256a55975371926: Status 404 returned error can't find the container with id ab966f9127d9cf2c1c21785d6db73d4f9708813f889962d6d256a55975371926
Aug 03 23:29:55.709250 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:55.709217    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcecf4ad_6c5e_4167_b617_5a9cc6015a24.slice/crio-109ad79424551fba40a12631adc9437c6141a862e77db8c641a75f0d02470fe7 WatchSource:0}: Error finding container 109ad79424551fba40a12631adc9437c6141a862e77db8c641a75f0d02470fe7: Status 404 returned error can't find the container with id 109ad79424551fba40a12631adc9437c6141a862e77db8c641a75f0d02470fe7
Aug 03 23:29:55.729564 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:55.729539    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a69b3b6_18d1_4f41_8529_ad97331b0aa6.slice/crio-8c5f62ad75f5f26fefefc858a6bb6a8a74bb3f6cf96a351b3a0ac8ab94879769 WatchSource:0}: Error finding container 8c5f62ad75f5f26fefefc858a6bb6a8a74bb3f6cf96a351b3a0ac8ab94879769: Status 404 returned error can't find the container with id 8c5f62ad75f5f26fefefc858a6bb6a8a74bb3f6cf96a351b3a0ac8ab94879769
Aug 03 23:29:55.730862 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:55.730842    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod418e085f_c3bf_46eb_9ce1_9751c038c924.slice/crio-b0fdddc339431961ddc02dd9ee3fea13f4bc89233d365c2a3b55c7989b3d5285 WatchSource:0}: Error finding container b0fdddc339431961ddc02dd9ee3fea13f4bc89233d365c2a3b55c7989b3d5285: Status 404 returned error can't find the container with id b0fdddc339431961ddc02dd9ee3fea13f4bc89233d365c2a3b55c7989b3d5285
Aug 03 23:29:55.731796 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:55.731764    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e167335_fef0_4d29_bfb7_8d4271a429e6.slice/crio-787906460a21e14ae1821a50e6284186697688d25f5b671cf3a719eb0bd8b069 WatchSource:0}: Error finding container 787906460a21e14ae1821a50e6284186697688d25f5b671cf3a719eb0bd8b069: Status 404 returned error can't find the container with id 787906460a21e14ae1821a50e6284186697688d25f5b671cf3a719eb0bd8b069
Aug 03 23:29:55.732477 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:29:55.732319    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce4267c2_aafd_422e_b7cd_a21565cebf5b.slice/crio-09856724a9dc9868787951efd9bc0add898c18f37ce8acbbef6b172ed7087494 WatchSource:0}: Error finding container 09856724a9dc9868787951efd9bc0add898c18f37ce8acbbef6b172ed7087494: Status 404 returned error can't find the container with id 09856724a9dc9868787951efd9bc0add898c18f37ce8acbbef6b172ed7087494
Aug 03 23:29:55.757848 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.757828    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:29:55.757962 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:55.757928    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:29:55.765541 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.765520    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-42jl2" event={"ID":"fcecf4ad-6c5e-4167-b617-5a9cc6015a24","Type":"ContainerStarted","Data":"109ad79424551fba40a12631adc9437c6141a862e77db8c641a75f0d02470fe7"}
Aug 03 23:29:55.766923 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.766898    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcf2p" event={"ID":"39619f91-b9cc-4d59-bcac-b0da494614ee","Type":"ContainerStarted","Data":"ab966f9127d9cf2c1c21785d6db73d4f9708813f889962d6d256a55975371926"}
Aug 03 23:29:55.769812 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.769788    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qm6s6" event={"ID":"ced51f35-5c98-4126-a3b4-533675741b86","Type":"ContainerStarted","Data":"0266788ec9baa8835b5f1904097248160c7a09ca2820318bf4db443cb2577943"}
Aug 03 23:29:55.775106 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.775084    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rhsgn" event={"ID":"8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c","Type":"ContainerStarted","Data":"50923380f8c7bd22064286655bb5d6d2364077e24c523b7330ac0b11a841ed2b"}
Aug 03 23:29:55.775962 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.775934    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g24wc" event={"ID":"418e085f-c3bf-46eb-9ce1-9751c038c924","Type":"ContainerStarted","Data":"b0fdddc339431961ddc02dd9ee3fea13f4bc89233d365c2a3b55c7989b3d5285"}
Aug 03 23:29:55.776830 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.776811    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkhxt" event={"ID":"79fe88bf-1fbb-4616-9dbb-3a3ccab24265","Type":"ContainerStarted","Data":"9489f3945cf88d93e0713862c6f665e1ca0df4782ff10ae033a97c20d04da38b"}
Aug 03 23:29:55.780380 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.780359    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5" event={"ID":"ce4267c2-aafd-422e-b7cd-a21565cebf5b","Type":"ContainerStarted","Data":"09856724a9dc9868787951efd9bc0add898c18f37ce8acbbef6b172ed7087494"}
Aug 03 23:29:55.781919 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.781865    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh" event={"ID":"3e167335-fef0-4d29-bfb7-8d4271a429e6","Type":"ContainerStarted","Data":"787906460a21e14ae1821a50e6284186697688d25f5b671cf3a719eb0bd8b069"}
Aug 03 23:29:55.783070 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:55.783049    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cw2jt" event={"ID":"6a69b3b6-18d1-4f41-8529-ad97331b0aa6","Type":"ContainerStarted","Data":"8c5f62ad75f5f26fefefc858a6bb6a8a74bb3f6cf96a351b3a0ac8ab94879769"}
Aug 03 23:29:56.359571 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:56.359283    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:29:56.359751 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:56.359589    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:56.359751 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:56.359453    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:29:56.359874 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:56.359786    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs podName:5e420d03-b0de-4f89-936a-cdafff2847a9 nodeName:}" failed. No retries permitted until 2026-08-03 23:29:58.359767399 +0000 UTC m=+6.061474064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs") pod "network-metrics-daemon-kld8m" (UID: "5e420d03-b0de-4f89-936a-cdafff2847a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:29:56.360113 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:56.359724    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 23:29:56.360204 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:56.360153    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret podName:57500f92-3fda-430a-bbc0-37e3d091d27f nodeName:}" failed. No retries permitted until 2026-08-03 23:29:58.36013744 +0000 UTC m=+6.061844101 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret") pod "global-pull-secret-syncer-w526z" (UID: "57500f92-3fda-430a-bbc0-37e3d091d27f") : object "kube-system"/"original-pull-secret" not registered
Aug 03 23:29:56.460396 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:56.460207    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfr8d\" (UniqueName: \"kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d\") pod \"network-check-target-pfdwg\" (UID: \"971b4204-55a9-4942-8e3c-c202fa0b93e2\") " pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:29:56.460396 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:56.460380    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 23:29:56.460396 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:56.460399    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 23:29:56.460642 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:56.460412    2553 projected.go:196] Error preparing data for projected volume kube-api-access-sfr8d for pod openshift-network-diagnostics/network-check-target-pfdwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:29:56.460642 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:56.460464    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d podName:971b4204-55a9-4942-8e3c-c202fa0b93e2 nodeName:}" failed. No retries permitted until 2026-08-03 23:29:58.46044855 +0000 UTC m=+6.162155196 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfr8d" (UniqueName: "kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d") pod "network-check-target-pfdwg" (UID: "971b4204-55a9-4942-8e3c-c202fa0b93e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:29:56.680640 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:56.680509    2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-02 23:24:54 +0000 UTC" deadline="2028-02-27 01:26:12.142139137 +0000 UTC"
Aug 03 23:29:56.680640 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:56.680569    2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13729h56m15.461574069s"
Aug 03 23:29:56.760471 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:56.759954    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:29:56.760471 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:56.760090    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:29:56.761082 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:56.760660    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:56.761082 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:56.760807    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:29:56.807920 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:56.807890    2553 generic.go:356] "Generic (PLEG): container finished" podID="99cb8e83d430af053d61a47269a3a9b8" containerID="28161be2a87f7778f99fa78b31b1032b5b0e13011cad0775e6fce6b00dd4583e" exitCode=0
Aug 03 23:29:56.808292 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:56.808225    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal" event={"ID":"99cb8e83d430af053d61a47269a3a9b8","Type":"ContainerDied","Data":"28161be2a87f7778f99fa78b31b1032b5b0e13011cad0775e6fce6b00dd4583e"}
Aug 03 23:29:56.818122 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:56.818096    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-183.ec2.internal" event={"ID":"43e10beb2c51db4c48da59eafdb1be9c","Type":"ContainerStarted","Data":"2517e5661102394fb6fec7bedcf17c62e40cab426f70809bbfae9703db2f7b0a"}
Aug 03 23:29:57.758628 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:57.758600    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:29:57.758796 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:57.758739    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:29:57.849975 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:57.849078    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal" event={"ID":"99cb8e83d430af053d61a47269a3a9b8","Type":"ContainerStarted","Data":"91cfc82bf0fefcdf55a3f09a0aa03eddba7e60782c3b43629c7ed6715acd9829"}
Aug 03 23:29:57.864385 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:57.864324    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-183.ec2.internal" podStartSLOduration=4.864305018 podStartE2EDuration="4.864305018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:29:56.83826605 +0000 UTC m=+4.539972714" watchObservedRunningTime="2026-08-03 23:29:57.864305018 +0000 UTC m=+5.566011684"
Aug 03 23:29:58.377937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:58.377281    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:58.377937 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:58.377349    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:29:58.377937 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:58.377486    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:29:58.377937 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:58.377545    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs podName:5e420d03-b0de-4f89-936a-cdafff2847a9 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:02.377527391 +0000 UTC m=+10.079234039 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs") pod "network-metrics-daemon-kld8m" (UID: "5e420d03-b0de-4f89-936a-cdafff2847a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:29:58.377937 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:58.377818    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 23:29:58.377937 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:58.377864    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret podName:57500f92-3fda-430a-bbc0-37e3d091d27f nodeName:}" failed. No retries permitted until 2026-08-03 23:30:02.377850673 +0000 UTC m=+10.079557316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret") pod "global-pull-secret-syncer-w526z" (UID: "57500f92-3fda-430a-bbc0-37e3d091d27f") : object "kube-system"/"original-pull-secret" not registered
Aug 03 23:29:58.478651 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:58.478615    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfr8d\" (UniqueName: \"kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d\") pod \"network-check-target-pfdwg\" (UID: \"971b4204-55a9-4942-8e3c-c202fa0b93e2\") " pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:29:58.478859 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:58.478842    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 23:29:58.478929 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:58.478868    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 23:29:58.478929 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:58.478881    2553 projected.go:196] Error preparing data for projected volume kube-api-access-sfr8d for pod openshift-network-diagnostics/network-check-target-pfdwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:29:58.479031 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:58.478960    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d podName:971b4204-55a9-4942-8e3c-c202fa0b93e2 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:02.478940699 +0000 UTC m=+10.180647342 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfr8d" (UniqueName: "kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d") pod "network-check-target-pfdwg" (UID: "971b4204-55a9-4942-8e3c-c202fa0b93e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:29:58.758891 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:58.758818    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:29:58.759049 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:58.758956    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:29:58.759391 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:58.759371    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:29:58.759489 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:58.759468    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:29:59.758407 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:29:59.757947    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:29:59.758407 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:29:59.758042    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:00.758292 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:00.758253    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:00.758462 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:00.758384    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:00.758834 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:00.758257    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:00.758834 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:00.758763    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:01.757830 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:01.757797    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:01.758006 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:01.757931    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:02.409859 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:02.409818    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:02.410255 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:02.409882    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:02.410255 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:02.410026    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:30:02.410255 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:02.410082    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs podName:5e420d03-b0de-4f89-936a-cdafff2847a9 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:10.410063928 +0000 UTC m=+18.111770584 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs") pod "network-metrics-daemon-kld8m" (UID: "5e420d03-b0de-4f89-936a-cdafff2847a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:30:02.410545 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:02.410520    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 23:30:02.410621 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:02.410573    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret podName:57500f92-3fda-430a-bbc0-37e3d091d27f nodeName:}" failed. No retries permitted until 2026-08-03 23:30:10.41055802 +0000 UTC m=+18.112264676 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret") pod "global-pull-secret-syncer-w526z" (UID: "57500f92-3fda-430a-bbc0-37e3d091d27f") : object "kube-system"/"original-pull-secret" not registered
Aug 03 23:30:02.510841 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:02.510807    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfr8d\" (UniqueName: \"kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d\") pod \"network-check-target-pfdwg\" (UID: \"971b4204-55a9-4942-8e3c-c202fa0b93e2\") " pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:02.511008 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:02.510964    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 23:30:02.511008 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:02.510984    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 23:30:02.511008 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:02.510997    2553 projected.go:196] Error preparing data for projected volume kube-api-access-sfr8d for pod openshift-network-diagnostics/network-check-target-pfdwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:30:02.511180 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:02.511047    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d podName:971b4204-55a9-4942-8e3c-c202fa0b93e2 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:10.511029987 +0000 UTC m=+18.212736644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfr8d" (UniqueName: "kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d") pod "network-check-target-pfdwg" (UID: "971b4204-55a9-4942-8e3c-c202fa0b93e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:30:02.764897 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:02.764818    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:02.765057 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:02.764949    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:02.765235 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:02.765215    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:02.765452 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:02.765424    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:03.758632 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:03.758601    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:03.759141 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:03.758725    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:04.758230 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:04.758199    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:04.758405 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:04.758339    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:04.758471 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:04.758417    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:04.758554 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:04.758530    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:05.758039 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:05.758011    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:05.758434 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:05.758108    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:06.758670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:06.758566    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:06.758670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:06.758583    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:06.759164 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:06.758726    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:06.759164 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:06.759155    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:07.758299 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:07.758265    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:07.758478 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:07.758385    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:08.761580 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:08.761551    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:08.762041 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:08.761548    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:08.762041 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:08.761685    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:08.762041 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:08.761776    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:09.758439 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:09.758410    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:09.758680 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:09.758540    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:10.471121 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:10.471090    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:10.471555 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:10.471133    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:10.471555 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:10.471240    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:30:10.471555 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:10.471240    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 23:30:10.471555 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:10.471301    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret podName:57500f92-3fda-430a-bbc0-37e3d091d27f nodeName:}" failed. No retries permitted until 2026-08-03 23:30:26.471284131 +0000 UTC m=+34.172990773 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret") pod "global-pull-secret-syncer-w526z" (UID: "57500f92-3fda-430a-bbc0-37e3d091d27f") : object "kube-system"/"original-pull-secret" not registered
Aug 03 23:30:10.471555 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:10.471317    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs podName:5e420d03-b0de-4f89-936a-cdafff2847a9 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:26.471308537 +0000 UTC m=+34.173015179 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs") pod "network-metrics-daemon-kld8m" (UID: "5e420d03-b0de-4f89-936a-cdafff2847a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:30:10.571931 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:10.571907    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfr8d\" (UniqueName: \"kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d\") pod \"network-check-target-pfdwg\" (UID: \"971b4204-55a9-4942-8e3c-c202fa0b93e2\") " pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:10.572030 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:10.572018    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 23:30:10.572068 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:10.572037    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 23:30:10.572068 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:10.572046    2553 projected.go:196] Error preparing data for projected volume kube-api-access-sfr8d for pod openshift-network-diagnostics/network-check-target-pfdwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:30:10.572129 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:10.572086    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d podName:971b4204-55a9-4942-8e3c-c202fa0b93e2 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:26.572074833 +0000 UTC m=+34.273781475 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfr8d" (UniqueName: "kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d") pod "network-check-target-pfdwg" (UID: "971b4204-55a9-4942-8e3c-c202fa0b93e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:30:10.761615 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:10.761545    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:10.761843 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:10.761550    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:10.761843 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:10.761671    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:10.761843 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:10.761766    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:11.757852 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:11.757823    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:11.758224 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:11.757920    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:12.758723 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:12.758672    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:12.759094 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:12.758778    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:12.759094 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:12.758793    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:12.759094 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:12.758899    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:13.758529 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:13.758492    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:13.758695 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:13.758627    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:14.761285 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.761018    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:14.761809 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.761086    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:14.761809 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:14.761363    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:14.761809 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:14.761429    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:14.881891 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.881862    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g24wc" event={"ID":"418e085f-c3bf-46eb-9ce1-9751c038c924","Type":"ContainerStarted","Data":"43e9b34415a4efc5238025c38bfca74afbde97b27ba0049cec6efe2d2e049a42"}
Aug 03 23:30:14.883399 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.883373    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkhxt" event={"ID":"79fe88bf-1fbb-4616-9dbb-3a3ccab24265","Type":"ContainerStarted","Data":"073d8d75079d0570105d51349fe23679b1623dd0de4506f894c1e64da9db967b"}
Aug 03 23:30:14.885841 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.885817    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5" event={"ID":"ce4267c2-aafd-422e-b7cd-a21565cebf5b","Type":"ContainerStarted","Data":"972a32bee09ebdc83e52f2dda4b9aff45b9065e731c1789237f769e6523ffa5b"}
Aug 03 23:30:14.885931 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.885892    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5" event={"ID":"ce4267c2-aafd-422e-b7cd-a21565cebf5b","Type":"ContainerStarted","Data":"7e346bb557d7a4245d8f8f0c1d9b835c55fef811b2c629edfd9f3173230ea7cd"}
Aug 03 23:30:14.885981 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.885929    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5" event={"ID":"ce4267c2-aafd-422e-b7cd-a21565cebf5b","Type":"ContainerStarted","Data":"d4b287f3094e05d24a679b98829ea3c20e233588d2316162ed419284a2b90807"}
Aug 03 23:30:14.885981 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.885965    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5" event={"ID":"ce4267c2-aafd-422e-b7cd-a21565cebf5b","Type":"ContainerStarted","Data":"cfa13cc2524ff599862a08b7110f99c4d82e3fa5bcd745f9d0e1af551f3ee144"}
Aug 03 23:30:14.887159 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.887143    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh" event={"ID":"3e167335-fef0-4d29-bfb7-8d4271a429e6","Type":"ContainerStarted","Data":"320e0720e868b26b2c9ec698e48e124530e132f3156d20e392faa5a388075ca6"}
Aug 03 23:30:14.888297 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.888278    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cw2jt" event={"ID":"6a69b3b6-18d1-4f41-8529-ad97331b0aa6","Type":"ContainerStarted","Data":"b4d5201ce263ee4ec85f6bf217568d236c57c0803438678cc7f7db6073dead46"}
Aug 03 23:30:14.889523 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.889505    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-42jl2" event={"ID":"fcecf4ad-6c5e-4167-b617-5a9cc6015a24","Type":"ContainerStarted","Data":"09de010fc8bc4ad56a0626a82d1d43d065fe60c0c2f15c143fd8a5e0abb14479"}
Aug 03 23:30:14.890861 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.890843    2553 generic.go:356] "Generic (PLEG): container finished" podID="39619f91-b9cc-4d59-bcac-b0da494614ee" containerID="b79cff97bf45bcc56ea9122c7287633c2073f4661a858cc09c90ef993a226eb8" exitCode=0
Aug 03 23:30:14.890947 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.890904    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcf2p" event={"ID":"39619f91-b9cc-4d59-bcac-b0da494614ee","Type":"ContainerDied","Data":"b79cff97bf45bcc56ea9122c7287633c2073f4661a858cc09c90ef993a226eb8"}
Aug 03 23:30:14.892130 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.892114    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qm6s6" event={"ID":"ced51f35-5c98-4126-a3b4-533675741b86","Type":"ContainerStarted","Data":"4f8b299e667325f1b2e5b1e753d324650f8d9d44bac77c9df81669d54617b13b"}
Aug 03 23:30:14.899463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.899428    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-183.ec2.internal" podStartSLOduration=21.899417815 podStartE2EDuration="21.899417815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:29:57.865347521 +0000 UTC m=+5.567054189" watchObservedRunningTime="2026-08-03 23:30:14.899417815 +0000 UTC m=+22.601124479"
Aug 03 23:30:14.899565 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.899547    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g24wc" podStartSLOduration=4.262411436 podStartE2EDuration="22.899541815s" totalImagesPullingTime="18.637130379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:30:14.898007343 +0000 UTC m=+22.599714009" watchObservedRunningTime="2026-08-03 23:30:14.899541815 +0000 UTC m=+22.601248523"
Aug 03 23:30:14.911284 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.911242    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cw2jt" podStartSLOduration=8.085051525 podStartE2EDuration="22.911230689s" totalImagesPullingTime="14.826179164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:30:14.911094176 +0000 UTC m=+22.612800842" watchObservedRunningTime="2026-08-03 23:30:14.911230689 +0000 UTC m=+22.612937353"
Aug 03 23:30:14.927430 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.927385    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-42jl2" podStartSLOduration=4.333245975 podStartE2EDuration="22.927368163s" totalImagesPullingTime="18.594122188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:30:14.927013667 +0000 UTC m=+22.628720332" watchObservedRunningTime="2026-08-03 23:30:14.927368163 +0000 UTC m=+22.629074828"
Aug 03 23:30:14.963177 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.963123    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qm6s6" podStartSLOduration=4.3718567329999996 podStartE2EDuration="22.963109031s" totalImagesPullingTime="18.591252298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:30:14.963032616 +0000 UTC m=+22.664739271" watchObservedRunningTime="2026-08-03 23:30:14.963109031 +0000 UTC m=+22.664815695"
Aug 03 23:30:14.977914 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:14.977871    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wkhxt" podStartSLOduration=4.405354079 podStartE2EDuration="22.977851979s" totalImagesPullingTime="18.5724979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:30:14.977758563 +0000 UTC m=+22.679465232" watchObservedRunningTime="2026-08-03 23:30:14.977851979 +0000 UTC m=+22.679558677"
Aug 03 23:30:15.738440 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:15.738413    2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 03 23:30:15.758031 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:15.758005    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:15.758142 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:15.758121    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:15.867260 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:15.867182    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:30:15.897540 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:15.897508    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5" event={"ID":"ce4267c2-aafd-422e-b7cd-a21565cebf5b","Type":"ContainerStarted","Data":"a5cef087c4df782fc71f57de749ffb4b420013b2d89eb6fc45c3a3611ced2f6f"}
Aug 03 23:30:15.897677 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:15.897586    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5" event={"ID":"ce4267c2-aafd-422e-b7cd-a21565cebf5b","Type":"ContainerStarted","Data":"6278764ea5a2a4eaec333b9f5815c0911e262edf1579bf7336f578113a0ca8f3"}
Aug 03 23:30:15.899439 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:15.899410    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh" event={"ID":"3e167335-fef0-4d29-bfb7-8d4271a429e6","Type":"ContainerStarted","Data":"31125ed3172e1b68161d473622adcf40c7d295e872fc7bf9a4ebdf759798c044"}
Aug 03 23:30:15.901412 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:15.900953    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rhsgn" event={"ID":"8dfbd7e7-129f-4bf3-a9ea-3fb006b2a64c","Type":"ContainerStarted","Data":"cac18ee09b1a24445a3d2939378b8aed57f7bf81ccde95a341c4eefbd7ec2b66"}
Aug 03 23:30:16.669788 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:16.669747    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:30:16.670501 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:16.670479    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:30:16.684840 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:16.684791    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rhsgn" podStartSLOduration=6.088200807 podStartE2EDuration="24.684772277s" totalImagesPullingTime="18.59657147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:30:15.915045156 +0000 UTC m=+23.616751819" watchObservedRunningTime="2026-08-03 23:30:16.684772277 +0000 UTC m=+24.386478941"
Aug 03 23:30:16.705909 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:16.705765    2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-03T23:30:15.738434967Z","UUID":"19fa636b-2cb5-4979-a3fb-9e9fbf7b2a2d","Handler":null,"Name":"","Endpoint":""}
Aug 03 23:30:16.707617 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:16.707599    2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 03 23:30:16.707743 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:16.707639    2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 03 23:30:16.758361 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:16.758333    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:16.758493 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:16.758459    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:16.758563 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:16.758496    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:16.758617 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:16.758592    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:16.906160 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:16.906125    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh" event={"ID":"3e167335-fef0-4d29-bfb7-8d4271a429e6","Type":"ContainerStarted","Data":"33b5ed8199c0761ec746f0c121ce3400c2f595259296ed0b798e324fdcd27e23"}
Aug 03 23:30:16.906806 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:16.906784    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cw2jt"
Aug 03 23:30:17.758162 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:17.758134    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:17.758376 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:17.758235    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:17.911768 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:17.911724    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5" event={"ID":"ce4267c2-aafd-422e-b7cd-a21565cebf5b","Type":"ContainerStarted","Data":"f4b6e26564a15f25934717b684f1a239b537ec85287d9753b230a60402fb35d9"}
Aug 03 23:30:17.929522 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:17.929470    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hjwh" podStartSLOduration=5.249279203 podStartE2EDuration="25.929454906s" totalImagesPullingTime="20.680175703s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:29:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:30:17.928259017 +0000 UTC m=+25.629965712" watchObservedRunningTime="2026-08-03 23:30:17.929454906 +0000 UTC m=+25.631161571"
Aug 03 23:30:18.758310 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:18.758280    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:18.758476 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:18.758413    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:18.758476 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:18.758461    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:18.758611 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:18.758574    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:19.758466 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:19.758313    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:19.759030 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:19.758538    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:19.917798 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:19.917772    2553 generic.go:356] "Generic (PLEG): container finished" podID="39619f91-b9cc-4d59-bcac-b0da494614ee" containerID="91d6dcdfa9d4b01113d4eebb5e66df4d5c35cdd7b5e2ac6036eeb6f8def2c0a6" exitCode=0
Aug 03 23:30:19.917905 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:19.917869    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcf2p" event={"ID":"39619f91-b9cc-4d59-bcac-b0da494614ee","Type":"ContainerDied","Data":"91d6dcdfa9d4b01113d4eebb5e66df4d5c35cdd7b5e2ac6036eeb6f8def2c0a6"}
Aug 03 23:30:19.921458 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:19.921437    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5" event={"ID":"ce4267c2-aafd-422e-b7cd-a21565cebf5b","Type":"ContainerStarted","Data":"9ec4e2c45e620eb8810c9446d76e1a744a1f05f55abcfecf0d96430bcf5a06d8"}
Aug 03 23:30:19.921725 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:19.921690    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:30:19.921802 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:19.921757    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:30:19.921802 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:19.921799    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:30:19.936742 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:19.936701    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:30:19.936848 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:19.936836    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:30:19.967653 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:19.967410    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5" podStartSLOduration=9.313045903 podStartE2EDuration="27.967393009s" totalImagesPullingTime="18.654347106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:30:19.966247396 +0000 UTC m=+27.667954054" watchObservedRunningTime="2026-08-03 23:30:19.967393009 +0000 UTC m=+27.669099674"
Aug 03 23:30:20.758426 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:20.758398    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:20.758640 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:20.758506    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:20.758640 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:20.758559    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:20.758967 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:20.758694    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:21.445172 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:21.445139    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kld8m"]
Aug 03 23:30:21.445881 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:21.445259    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:21.445881 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:21.445380    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:21.446011 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:21.445990    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w526z"]
Aug 03 23:30:21.446082 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:21.446069    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:21.446198 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:21.446168    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:21.446701 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:21.446677    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pfdwg"]
Aug 03 23:30:21.446818 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:21.446791    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:21.446906 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:21.446881    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:21.927434 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:21.927400    2553 generic.go:356] "Generic (PLEG): container finished" podID="39619f91-b9cc-4d59-bcac-b0da494614ee" containerID="436696136fbd229a09c434379cb69c808d4b902bcc608ba38cc6f36713b28518" exitCode=0
Aug 03 23:30:21.927434 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:21.927432    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcf2p" event={"ID":"39619f91-b9cc-4d59-bcac-b0da494614ee","Type":"ContainerDied","Data":"436696136fbd229a09c434379cb69c808d4b902bcc608ba38cc6f36713b28518"}
Aug 03 23:30:22.759202 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:22.759169    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:22.759409 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:22.759259    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:22.759409 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:22.759307    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:22.759409 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:22.759398    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:23.758681 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:23.758611    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:23.759110 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:23.758736    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:23.933726 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:23.933675    2553 generic.go:356] "Generic (PLEG): container finished" podID="39619f91-b9cc-4d59-bcac-b0da494614ee" containerID="3c6668671aba6a94289b6c22f3a92244bd61942bde0e2c1f4e097c155255dd6e" exitCode=0
Aug 03 23:30:23.933872 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:23.933743    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcf2p" event={"ID":"39619f91-b9cc-4d59-bcac-b0da494614ee","Type":"ContainerDied","Data":"3c6668671aba6a94289b6c22f3a92244bd61942bde0e2c1f4e097c155255dd6e"}
Aug 03 23:30:24.758403 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:24.758369    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:24.758579 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:24.758375    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:24.758579 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:24.758500    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:24.758727 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:24.758687    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:25.758106 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:25.758073    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:25.758289 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:25.758184    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:26.498098 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:26.498064    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:26.498531 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:26.498137    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:26.498531 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:26.498191    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:30:26.498531 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:26.498249    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 23:30:26.498531 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:26.498264    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs podName:5e420d03-b0de-4f89-936a-cdafff2847a9 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:58.498242662 +0000 UTC m=+66.199949327 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs") pod "network-metrics-daemon-kld8m" (UID: "5e420d03-b0de-4f89-936a-cdafff2847a9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 23:30:26.498531 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:26.498297    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret podName:57500f92-3fda-430a-bbc0-37e3d091d27f nodeName:}" failed. No retries permitted until 2026-08-03 23:30:58.498281706 +0000 UTC m=+66.199988349 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret") pod "global-pull-secret-syncer-w526z" (UID: "57500f92-3fda-430a-bbc0-37e3d091d27f") : object "kube-system"/"original-pull-secret" not registered
Aug 03 23:30:26.598504 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:26.598473    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfr8d\" (UniqueName: \"kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d\") pod \"network-check-target-pfdwg\" (UID: \"971b4204-55a9-4942-8e3c-c202fa0b93e2\") " pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:26.598666 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:26.598648    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 23:30:26.598763 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:26.598673    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 23:30:26.598763 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:26.598686    2553 projected.go:196] Error preparing data for projected volume kube-api-access-sfr8d for pod openshift-network-diagnostics/network-check-target-pfdwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:30:26.598862 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:26.598768    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d podName:971b4204-55a9-4942-8e3c-c202fa0b93e2 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:58.598749117 +0000 UTC m=+66.300455774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfr8d" (UniqueName: "kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d") pod "network-check-target-pfdwg" (UID: "971b4204-55a9-4942-8e3c-c202fa0b93e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 23:30:26.758220 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:26.758145    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:26.758373 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:26.758154    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:26.758373 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:26.758274    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:30:26.758373 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:26.758341    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w526z" podUID="57500f92-3fda-430a-bbc0-37e3d091d27f"
Aug 03 23:30:27.758127 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:27.758094    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:27.758549 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:27.758214    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pfdwg" podUID="971b4204-55a9-4942-8e3c-c202fa0b93e2"
Aug 03 23:30:28.089134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.088936    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-183.ec2.internal" event="NodeReady"
Aug 03 23:30:28.089285 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.089269    2553 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 03 23:30:28.132432 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.132406    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6mrf2"]
Aug 03 23:30:28.134641 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.134625    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-np8h4"]
Aug 03 23:30:28.134772 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.134757    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.136257 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.136236    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-np8h4"
Aug 03 23:30:28.136995 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.136977    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 03 23:30:28.137075 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.136981    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 03 23:30:28.137075 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.136981    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rmmw5\""
Aug 03 23:30:28.138600 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.138582    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4skmh\""
Aug 03 23:30:28.138702 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.138653    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 03 23:30:28.138818 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.138802    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 03 23:30:28.138881 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.138820    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 03 23:30:28.142845 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.142801    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-np8h4"]
Aug 03 23:30:28.145626 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.145509    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6mrf2"]
Aug 03 23:30:28.312114 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.312092    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.312185 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.312120    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/731c0106-b5c5-4a64-957d-38254cee1862-tmp-dir\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.312185 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.312139    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxf9m\" (UniqueName: \"kubernetes.io/projected/731c0106-b5c5-4a64-957d-38254cee1862-kube-api-access-gxf9m\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.312288 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.312247    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/966fe46a-d62f-4d39-ae52-5e9e12b681c6-cert\") pod \"ingress-canary-np8h4\" (UID: \"966fe46a-d62f-4d39-ae52-5e9e12b681c6\") " pod="openshift-ingress-canary/ingress-canary-np8h4"
Aug 03 23:30:28.312288 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.312277    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/731c0106-b5c5-4a64-957d-38254cee1862-config-volume\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.312345 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.312302    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgwbk\" (UniqueName: \"kubernetes.io/projected/966fe46a-d62f-4d39-ae52-5e9e12b681c6-kube-api-access-kgwbk\") pod \"ingress-canary-np8h4\" (UID: \"966fe46a-d62f-4d39-ae52-5e9e12b681c6\") " pod="openshift-ingress-canary/ingress-canary-np8h4"
Aug 03 23:30:28.413382 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.413362    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/966fe46a-d62f-4d39-ae52-5e9e12b681c6-cert\") pod \"ingress-canary-np8h4\" (UID: \"966fe46a-d62f-4d39-ae52-5e9e12b681c6\") " pod="openshift-ingress-canary/ingress-canary-np8h4"
Aug 03 23:30:28.413481 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.413390    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/731c0106-b5c5-4a64-957d-38254cee1862-config-volume\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.413481 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.413409    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgwbk\" (UniqueName: \"kubernetes.io/projected/966fe46a-d62f-4d39-ae52-5e9e12b681c6-kube-api-access-kgwbk\") pod \"ingress-canary-np8h4\" (UID: \"966fe46a-d62f-4d39-ae52-5e9e12b681c6\") " pod="openshift-ingress-canary/ingress-canary-np8h4"
Aug 03 23:30:28.413602 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.413538    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.413602 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.413577    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/731c0106-b5c5-4a64-957d-38254cee1862-tmp-dir\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.413726 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.413601    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxf9m\" (UniqueName: \"kubernetes.io/projected/731c0106-b5c5-4a64-957d-38254cee1862-kube-api-access-gxf9m\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.413726 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:28.413684    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 23:30:28.413819 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:28.413784    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls podName:731c0106-b5c5-4a64-957d-38254cee1862 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:28.913763395 +0000 UTC m=+36.615470056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls") pod "dns-default-6mrf2" (UID: "731c0106-b5c5-4a64-957d-38254cee1862") : secret "dns-default-metrics-tls" not found
Aug 03 23:30:28.413920 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.413891    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/731c0106-b5c5-4a64-957d-38254cee1862-tmp-dir\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.414024 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.414006    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/731c0106-b5c5-4a64-957d-38254cee1862-config-volume\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.417828 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.417808    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/966fe46a-d62f-4d39-ae52-5e9e12b681c6-cert\") pod \"ingress-canary-np8h4\" (UID: \"966fe46a-d62f-4d39-ae52-5e9e12b681c6\") " pod="openshift-ingress-canary/ingress-canary-np8h4"
Aug 03 23:30:28.422213 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.422190    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgwbk\" (UniqueName: \"kubernetes.io/projected/966fe46a-d62f-4d39-ae52-5e9e12b681c6-kube-api-access-kgwbk\") pod \"ingress-canary-np8h4\" (UID: \"966fe46a-d62f-4d39-ae52-5e9e12b681c6\") " pod="openshift-ingress-canary/ingress-canary-np8h4"
Aug 03 23:30:28.422346 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.422331    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxf9m\" (UniqueName: \"kubernetes.io/projected/731c0106-b5c5-4a64-957d-38254cee1862-kube-api-access-gxf9m\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.474377 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.474348    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-np8h4"
Aug 03 23:30:28.601965 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.601936    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-np8h4"]
Aug 03 23:30:28.605578 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:30:28.605554    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod966fe46a_d62f_4d39_ae52_5e9e12b681c6.slice/crio-426d65292864ac84ca0b9c5266359bfd73b4b3cc3629bc07b51593d6897d44ee WatchSource:0}: Error finding container 426d65292864ac84ca0b9c5266359bfd73b4b3cc3629bc07b51593d6897d44ee: Status 404 returned error can't find the container with id 426d65292864ac84ca0b9c5266359bfd73b4b3cc3629bc07b51593d6897d44ee
Aug 03 23:30:28.758409 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.758347    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:28.759072 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.758505    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:28.760956 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.760910    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 03 23:30:28.761047 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.760960    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 03 23:30:28.761047 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.760963    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w6szp\""
Aug 03 23:30:28.918370 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.918351    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:28.918470 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:28.918444    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 23:30:28.918524 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:28.918514    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls podName:731c0106-b5c5-4a64-957d-38254cee1862 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:29.918495143 +0000 UTC m=+37.620201797 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls") pod "dns-default-6mrf2" (UID: "731c0106-b5c5-4a64-957d-38254cee1862") : secret "dns-default-metrics-tls" not found
Aug 03 23:30:28.947441 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.947415    2553 generic.go:356] "Generic (PLEG): container finished" podID="39619f91-b9cc-4d59-bcac-b0da494614ee" containerID="0529c10e807b45ae94d4316eb54e7708bb034db569488829055639d16aa81fea" exitCode=0
Aug 03 23:30:28.947530 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.947487    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcf2p" event={"ID":"39619f91-b9cc-4d59-bcac-b0da494614ee","Type":"ContainerDied","Data":"0529c10e807b45ae94d4316eb54e7708bb034db569488829055639d16aa81fea"}
Aug 03 23:30:28.948538 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:28.948519    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-np8h4" event={"ID":"966fe46a-d62f-4d39-ae52-5e9e12b681c6","Type":"ContainerStarted","Data":"426d65292864ac84ca0b9c5266359bfd73b4b3cc3629bc07b51593d6897d44ee"}
Aug 03 23:30:29.758971 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:29.758736    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:29.761647 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:29.761447    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 03 23:30:29.761647 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:29.761547    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 03 23:30:29.761647 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:29.761553    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n4wm4\""
Aug 03 23:30:29.927483 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:29.927451    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:29.927684 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:29.927608    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 23:30:29.927684 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:29.927668    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls podName:731c0106-b5c5-4a64-957d-38254cee1862 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:31.927649251 +0000 UTC m=+39.629355907 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls") pod "dns-default-6mrf2" (UID: "731c0106-b5c5-4a64-957d-38254cee1862") : secret "dns-default-metrics-tls" not found
Aug 03 23:30:29.954438 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:29.954353    2553 generic.go:356] "Generic (PLEG): container finished" podID="39619f91-b9cc-4d59-bcac-b0da494614ee" containerID="951f5da9dd32546f6f4a392dc7b9fc345b3429042eee60cd7fea6fdcf14c0c8c" exitCode=0
Aug 03 23:30:29.954438 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:29.954415    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcf2p" event={"ID":"39619f91-b9cc-4d59-bcac-b0da494614ee","Type":"ContainerDied","Data":"951f5da9dd32546f6f4a392dc7b9fc345b3429042eee60cd7fea6fdcf14c0c8c"}
Aug 03 23:30:30.960352 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:30.960322    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcf2p" event={"ID":"39619f91-b9cc-4d59-bcac-b0da494614ee","Type":"ContainerStarted","Data":"115e57b563996bd9f39f8d1e162288e91ae707b8d6b27b76f42ae0a8618f522d"}
Aug 03 23:30:30.984681 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:30.984637    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xcf2p" podStartSLOduration=9.418774749 podStartE2EDuration="38.984621549s" totalImagesPullingTime="29.5658468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:29:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:30:30.982891456 +0000 UTC m=+38.684598120" watchObservedRunningTime="2026-08-03 23:30:30.984621549 +0000 UTC m=+38.686328213"
Aug 03 23:30:31.943252 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:31.943216    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:31.943443 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:31.943384    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 23:30:31.943503 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:31.943449    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls podName:731c0106-b5c5-4a64-957d-38254cee1862 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:35.943434339 +0000 UTC m=+43.645140981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls") pod "dns-default-6mrf2" (UID: "731c0106-b5c5-4a64-957d-38254cee1862") : secret "dns-default-metrics-tls" not found
Aug 03 23:30:31.963880 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:31.963848    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-np8h4" event={"ID":"966fe46a-d62f-4d39-ae52-5e9e12b681c6","Type":"ContainerStarted","Data":"8b6b1a32915ee8ffb49648872ce76ed56363d2e3c14e17d579dd8740f6fcf7a7"}
Aug 03 23:30:31.978829 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:31.978654    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-np8h4" podStartSLOduration=1.5192327049999999 podStartE2EDuration="3.978636991s" totalImagesPullingTime="2.459404286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:30:31.978399534 +0000 UTC m=+39.680106200" watchObservedRunningTime="2026-08-03 23:30:31.978636991 +0000 UTC m=+39.680343634"
Aug 03 23:30:35.968631 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:35.968596    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:35.969000 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:35.968761    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 23:30:35.969000 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:35.968817    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls podName:731c0106-b5c5-4a64-957d-38254cee1862 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:43.968804156 +0000 UTC m=+51.670510801 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls") pod "dns-default-6mrf2" (UID: "731c0106-b5c5-4a64-957d-38254cee1862") : secret "dns-default-metrics-tls" not found
Aug 03 23:30:44.020353 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:44.020318    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:30:44.020834 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:44.020430    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 23:30:44.020834 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:44.020481    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls podName:731c0106-b5c5-4a64-957d-38254cee1862 nodeName:}" failed. No retries permitted until 2026-08-03 23:31:00.020467345 +0000 UTC m=+67.722173988 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls") pod "dns-default-6mrf2" (UID: "731c0106-b5c5-4a64-957d-38254cee1862") : secret "dns-default-metrics-tls" not found
Aug 03 23:30:45.268683 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.268653    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5fd5d454d-sfdvz"]
Aug 03 23:30:45.288920 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.288890    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5fd5d454d-sfdvz"]
Aug 03 23:30:45.289054 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.288985    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.291654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.291636    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 03 23:30:45.291916 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.291898    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m6v2t\""
Aug 03 23:30:45.292065 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.292046    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 03 23:30:45.292791 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.292775    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 03 23:30:45.297194 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.297176    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 03 23:30:45.328786 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.328761    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-trusted-ca\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.328874 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.328807    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.328874 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.328824    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5bg2\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-kube-api-access-m5bg2\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.328967 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.328876    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52d9b5aa-a776-405a-90a9-5341c61db6b2-ca-trust-extracted\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.328967 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.328911    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-certificates\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.328967 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.328951    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-image-registry-private-configuration\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.329065 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.328967    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-bound-sa-token\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.329065 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.328996    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-installation-pull-secrets\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.430186 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.430157    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52d9b5aa-a776-405a-90a9-5341c61db6b2-ca-trust-extracted\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.430277 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.430188    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-certificates\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.430323 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.430311    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-image-registry-private-configuration\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.430362 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.430333    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-bound-sa-token\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.430407 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.430362    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-installation-pull-secrets\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.430407 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.430391    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-trusted-ca\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.430570 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.430549    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.430646 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.430587    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5bg2\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-kube-api-access-m5bg2\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.430721 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.430559    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52d9b5aa-a776-405a-90a9-5341c61db6b2-ca-trust-extracted\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.430779 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:45.430728    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 23:30:45.430779 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:45.430746    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd5d454d-sfdvz: secret "image-registry-tls" not found
Aug 03 23:30:45.430920 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.430901    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-certificates\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.430998 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:45.430949    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls podName:52d9b5aa-a776-405a-90a9-5341c61db6b2 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:45.930932854 +0000 UTC m=+53.632639496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls") pod "image-registry-5fd5d454d-sfdvz" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2") : secret "image-registry-tls" not found
Aug 03 23:30:45.431300 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.431282    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-trusted-ca\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.434411 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.434395    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-installation-pull-secrets\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.434490 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.434470    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-image-registry-private-configuration\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.439151 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.439131    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-bound-sa-token\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.439721 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.439678    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5bg2\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-kube-api-access-m5bg2\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.933579 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:45.933554    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:45.933722 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:45.933674    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 23:30:45.933722 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:45.933685    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd5d454d-sfdvz: secret "image-registry-tls" not found
Aug 03 23:30:45.933802 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:45.933759    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls podName:52d9b5aa-a776-405a-90a9-5341c61db6b2 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:46.933744728 +0000 UTC m=+54.635451371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls") pod "image-registry-5fd5d454d-sfdvz" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2") : secret "image-registry-tls" not found
Aug 03 23:30:46.939590 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:46.939563    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:46.939990 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:46.939732    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 23:30:46.939990 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:46.939751    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd5d454d-sfdvz: secret "image-registry-tls" not found
Aug 03 23:30:46.939990 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:46.939809    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls podName:52d9b5aa-a776-405a-90a9-5341c61db6b2 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:48.939788196 +0000 UTC m=+56.641494837 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls") pod "image-registry-5fd5d454d-sfdvz" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2") : secret "image-registry-tls" not found
Aug 03 23:30:48.954272 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:48.954225    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:48.954795 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:48.954410    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 23:30:48.954795 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:48.954437    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd5d454d-sfdvz: secret "image-registry-tls" not found
Aug 03 23:30:48.954795 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:48.954524    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls podName:52d9b5aa-a776-405a-90a9-5341c61db6b2 nodeName:}" failed. No retries permitted until 2026-08-03 23:30:52.954503187 +0000 UTC m=+60.656209848 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls") pod "image-registry-5fd5d454d-sfdvz" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2") : secret "image-registry-tls" not found
Aug 03 23:30:51.949780 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:51.949755    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8bl5"
Aug 03 23:30:52.977978 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:52.977948    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:30:52.978309 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:52.978036    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 23:30:52.978309 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:52.978048    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd5d454d-sfdvz: secret "image-registry-tls" not found
Aug 03 23:30:52.978309 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:52.978100    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls podName:52d9b5aa-a776-405a-90a9-5341c61db6b2 nodeName:}" failed. No retries permitted until 2026-08-03 23:31:00.978087563 +0000 UTC m=+68.679794205 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls") pod "image-registry-5fd5d454d-sfdvz" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2") : secret "image-registry-tls" not found
Aug 03 23:30:58.511823 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.511789    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:30:58.512301 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.511856    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:58.514737 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.514699    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 03 23:30:58.514788 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.514726    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 03 23:30:58.522871 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:58.522846    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 03 23:30:58.522978 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:30:58.522928    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs podName:5e420d03-b0de-4f89-936a-cdafff2847a9 nodeName:}" failed. No retries permitted until 2026-08-03 23:32:02.522906643 +0000 UTC m=+130.224613299 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs") pod "network-metrics-daemon-kld8m" (UID: "5e420d03-b0de-4f89-936a-cdafff2847a9") : secret "metrics-daemon-secret" not found
Aug 03 23:30:58.525018 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.525000    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/57500f92-3fda-430a-bbc0-37e3d091d27f-original-pull-secret\") pod \"global-pull-secret-syncer-w526z\" (UID: \"57500f92-3fda-430a-bbc0-37e3d091d27f\") " pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:58.612715 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.612687    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfr8d\" (UniqueName: \"kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d\") pod \"network-check-target-pfdwg\" (UID: \"971b4204-55a9-4942-8e3c-c202fa0b93e2\") " pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:58.615252 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.615236    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 03 23:30:58.625418 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.625402    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 03 23:30:58.635611 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.635587    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfr8d\" (UniqueName: \"kubernetes.io/projected/971b4204-55a9-4942-8e3c-c202fa0b93e2-kube-api-access-sfr8d\") pod \"network-check-target-pfdwg\" (UID: \"971b4204-55a9-4942-8e3c-c202fa0b93e2\") " pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:58.768314 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.768266    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w526z"
Aug 03 23:30:58.871552 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.871530    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n4wm4\""
Aug 03 23:30:58.879781 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.879759    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:30:58.885842 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.885816    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w526z"]
Aug 03 23:30:58.889583 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:30:58.889506    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57500f92_3fda_430a_bbc0_37e3d091d27f.slice/crio-315fd39b8cf81ea2998eeed3eb0654db0ae325fe587f158ec252aca734fb8d36 WatchSource:0}: Error finding container 315fd39b8cf81ea2998eeed3eb0654db0ae325fe587f158ec252aca734fb8d36: Status 404 returned error can't find the container with id 315fd39b8cf81ea2998eeed3eb0654db0ae325fe587f158ec252aca734fb8d36
Aug 03 23:30:58.999143 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:58.999114    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pfdwg"]
Aug 03 23:30:59.002807 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:30:59.002752    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod971b4204_55a9_4942_8e3c_c202fa0b93e2.slice/crio-64a6a8482fe95a7005ae3f1436a282eff8f5840c2e15ca69d3e96725e1ce843a WatchSource:0}: Error finding container 64a6a8482fe95a7005ae3f1436a282eff8f5840c2e15ca69d3e96725e1ce843a: Status 404 returned error can't find the container with id 64a6a8482fe95a7005ae3f1436a282eff8f5840c2e15ca69d3e96725e1ce843a
Aug 03 23:30:59.024598 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:59.024540    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w526z" event={"ID":"57500f92-3fda-430a-bbc0-37e3d091d27f","Type":"ContainerStarted","Data":"315fd39b8cf81ea2998eeed3eb0654db0ae325fe587f158ec252aca734fb8d36"}
Aug 03 23:30:59.025451 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:30:59.025428    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pfdwg" event={"ID":"971b4204-55a9-4942-8e3c-c202fa0b93e2","Type":"ContainerStarted","Data":"64a6a8482fe95a7005ae3f1436a282eff8f5840c2e15ca69d3e96725e1ce843a"}
Aug 03 23:31:00.023551 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:00.023502    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:31:00.024041 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:00.023661    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 23:31:00.024041 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:00.023728    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls podName:731c0106-b5c5-4a64-957d-38254cee1862 nodeName:}" failed. No retries permitted until 2026-08-03 23:31:32.023697566 +0000 UTC m=+99.725404208 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls") pod "dns-default-6mrf2" (UID: "731c0106-b5c5-4a64-957d-38254cee1862") : secret "dns-default-metrics-tls" not found
Aug 03 23:31:01.031642 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:01.031615    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:31:01.032117 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:01.031772    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 23:31:01.032117 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:01.031787    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd5d454d-sfdvz: secret "image-registry-tls" not found
Aug 03 23:31:01.032117 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:01.031846    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls podName:52d9b5aa-a776-405a-90a9-5341c61db6b2 nodeName:}" failed. No retries permitted until 2026-08-03 23:31:17.031828351 +0000 UTC m=+84.733535007 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls") pod "image-registry-5fd5d454d-sfdvz" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2") : secret "image-registry-tls" not found
Aug 03 23:31:04.041600 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:04.041556    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pfdwg" event={"ID":"971b4204-55a9-4942-8e3c-c202fa0b93e2","Type":"ContainerStarted","Data":"f0568c71b765ad91ddc7e4b154836ecc47e937fe47046454a6e9b0dffd6bcf18"}
Aug 03 23:31:04.042135 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:04.041698    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:31:04.042946 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:04.042922    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w526z" event={"ID":"57500f92-3fda-430a-bbc0-37e3d091d27f","Type":"ContainerStarted","Data":"da1640f1e605074d4d0c01473f4d311264d4e4870ed7a674fc677485a4005f4d"}
Aug 03 23:31:04.057007 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:04.056964    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pfdwg" podStartSLOduration=67.504387044 podStartE2EDuration="1m12.056952073s" totalImagesPullingTime="4.552565029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:31:04.05680044 +0000 UTC m=+71.758507103" watchObservedRunningTime="2026-08-03 23:31:04.056952073 +0000 UTC m=+71.758658737"
Aug 03 23:31:04.071006 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:04.070970    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w526z" podStartSLOduration=66.403219735 podStartE2EDuration="1m11.070959009s" totalImagesPullingTime="4.667739274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:29:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:31:04.070475146 +0000 UTC m=+71.772181813" watchObservedRunningTime="2026-08-03 23:31:04.070959009 +0000 UTC m=+71.772665673"
Aug 03 23:31:17.036801 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:17.036676    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:31:17.037195 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:17.036801    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 23:31:17.037195 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:17.036815    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd5d454d-sfdvz: secret "image-registry-tls" not found
Aug 03 23:31:17.037195 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:17.036875    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls podName:52d9b5aa-a776-405a-90a9-5341c61db6b2 nodeName:}" failed. No retries permitted until 2026-08-03 23:31:49.036857872 +0000 UTC m=+116.738564516 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls") pod "image-registry-5fd5d454d-sfdvz" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2") : secret "image-registry-tls" not found
Aug 03 23:31:24.718635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.718604    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4"]
Aug 03 23:31:24.722742 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.722724    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4"
Aug 03 23:31:24.726303 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.726282    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Aug 03 23:31:24.726419 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.726395    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Aug 03 23:31:24.726419 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.726405    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Aug 03 23:31:24.726532 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.726445    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bht6v\""
Aug 03 23:31:24.726532 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.726490    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Aug 03 23:31:24.730402 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.730378    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4"]
Aug 03 23:31:24.788445 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.788424    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f4d89e2-b214-44da-b038-4bf0e6243726-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-659b947d7b-744n4\" (UID: \"6f4d89e2-b214-44da-b038-4bf0e6243726\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4"
Aug 03 23:31:24.788549 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.788492    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxwj8\" (UniqueName: \"kubernetes.io/projected/6f4d89e2-b214-44da-b038-4bf0e6243726-kube-api-access-zxwj8\") pod \"managed-serviceaccount-addon-agent-659b947d7b-744n4\" (UID: \"6f4d89e2-b214-44da-b038-4bf0e6243726\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4"
Aug 03 23:31:24.888942 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.888916    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxwj8\" (UniqueName: \"kubernetes.io/projected/6f4d89e2-b214-44da-b038-4bf0e6243726-kube-api-access-zxwj8\") pod \"managed-serviceaccount-addon-agent-659b947d7b-744n4\" (UID: \"6f4d89e2-b214-44da-b038-4bf0e6243726\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4"
Aug 03 23:31:24.889034 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.888955    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f4d89e2-b214-44da-b038-4bf0e6243726-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-659b947d7b-744n4\" (UID: \"6f4d89e2-b214-44da-b038-4bf0e6243726\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4"
Aug 03 23:31:24.891354 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.891337    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f4d89e2-b214-44da-b038-4bf0e6243726-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-659b947d7b-744n4\" (UID: \"6f4d89e2-b214-44da-b038-4bf0e6243726\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4"
Aug 03 23:31:24.897258 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:24.897237    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxwj8\" (UniqueName: \"kubernetes.io/projected/6f4d89e2-b214-44da-b038-4bf0e6243726-kube-api-access-zxwj8\") pod \"managed-serviceaccount-addon-agent-659b947d7b-744n4\" (UID: \"6f4d89e2-b214-44da-b038-4bf0e6243726\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4"
Aug 03 23:31:25.035815 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:25.035759    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4"
Aug 03 23:31:25.153415 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:25.153386    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4"]
Aug 03 23:31:25.156318 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:31:25.156288    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f4d89e2_b214_44da_b038_4bf0e6243726.slice/crio-d67367eb23cf486ad7da0476a9997bdd1792ad7b1fc1fc0f5180da4781666e7a WatchSource:0}: Error finding container d67367eb23cf486ad7da0476a9997bdd1792ad7b1fc1fc0f5180da4781666e7a: Status 404 returned error can't find the container with id d67367eb23cf486ad7da0476a9997bdd1792ad7b1fc1fc0f5180da4781666e7a
Aug 03 23:31:26.094905 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:26.094871    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4" event={"ID":"6f4d89e2-b214-44da-b038-4bf0e6243726","Type":"ContainerStarted","Data":"d67367eb23cf486ad7da0476a9997bdd1792ad7b1fc1fc0f5180da4781666e7a"}
Aug 03 23:31:30.105484 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:30.105448    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4" event={"ID":"6f4d89e2-b214-44da-b038-4bf0e6243726","Type":"ContainerStarted","Data":"e504fa1f460896f21aee1ae10674e54fa0dbeb205b8fbeaf1202c97b185bf4d5"}
Aug 03 23:31:30.120506 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:30.120465    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4" podStartSLOduration=2.193477846 podStartE2EDuration="6.120451057s" totalImagesPullingTime="3.926973211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:31:30.119752477 +0000 UTC m=+97.821459138" watchObservedRunningTime="2026-08-03 23:31:30.120451057 +0000 UTC m=+97.822157720"
Aug 03 23:31:32.039667 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:32.039637    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:31:32.040033 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:32.039811    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 23:31:32.040033 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:32.039876    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls podName:731c0106-b5c5-4a64-957d-38254cee1862 nodeName:}" failed. No retries permitted until 2026-08-03 23:32:36.039859786 +0000 UTC m=+163.741566427 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls") pod "dns-default-6mrf2" (UID: "731c0106-b5c5-4a64-957d-38254cee1862") : secret "dns-default-metrics-tls" not found
Aug 03 23:31:35.047434 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:35.047403    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pfdwg"
Aug 03 23:31:49.052572 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:31:49.052539    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:31:49.053069 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:49.052659    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 23:31:49.053069 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:49.052672    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd5d454d-sfdvz: secret "image-registry-tls" not found
Aug 03 23:31:49.053069 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:31:49.052749    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls podName:52d9b5aa-a776-405a-90a9-5341c61db6b2 nodeName:}" failed. No retries permitted until 2026-08-03 23:32:53.05273085 +0000 UTC m=+180.754437506 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls") pod "image-registry-5fd5d454d-sfdvz" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2") : secret "image-registry-tls" not found
Aug 03 23:32:02.542995 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:32:02.542963    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:32:02.543485 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:32:02.543115    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 03 23:32:02.543485 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:32:02.543190    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs podName:5e420d03-b0de-4f89-936a-cdafff2847a9 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:04.543173162 +0000 UTC m=+252.244879804 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs") pod "network-metrics-daemon-kld8m" (UID: "5e420d03-b0de-4f89-936a-cdafff2847a9") : secret "metrics-daemon-secret" not found
Aug 03 23:32:31.170126 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:32:31.170085    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6mrf2" podUID="731c0106-b5c5-4a64-957d-38254cee1862"
Aug 03 23:32:31.247000 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:32:31.246974    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:32:31.772850 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:32:31.772819    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:32:36.059641 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:32:36.059612    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:32:36.060019 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:32:36.059736    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 23:32:36.060019 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:32:36.059798    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls podName:731c0106-b5c5-4a64-957d-38254cee1862 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:38.05978422 +0000 UTC m=+285.761490864 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls") pod "dns-default-6mrf2" (UID: "731c0106-b5c5-4a64-957d-38254cee1862") : secret "dns-default-metrics-tls" not found
Aug 03 23:32:45.758434 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:32:45.758354    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:32:48.298816 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:32:48.298783    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz" podUID="52d9b5aa-a776-405a-90a9-5341c61db6b2"
Aug 03 23:32:53.066958 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:32:53.066917    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls\") pod \"image-registry-5fd5d454d-sfdvz\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") " pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:32:53.067413 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:32:53.067060    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 23:32:53.067413 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:32:53.067077    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5fd5d454d-sfdvz: secret "image-registry-tls" not found
Aug 03 23:32:53.067413 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:32:53.067143    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls podName:52d9b5aa-a776-405a-90a9-5341c61db6b2 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:55.067126703 +0000 UTC m=+302.768833344 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls") pod "image-registry-5fd5d454d-sfdvz" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2") : secret "image-registry-tls" not found
Aug 03 23:32:59.758086 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:32:59.758056    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:33:29.383286 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:33:29.383254    2553 generic.go:356] "Generic (PLEG): container finished" podID="6f4d89e2-b214-44da-b038-4bf0e6243726" containerID="e504fa1f460896f21aee1ae10674e54fa0dbeb205b8fbeaf1202c97b185bf4d5" exitCode=255
Aug 03 23:33:29.383612 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:33:29.383301    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4" event={"ID":"6f4d89e2-b214-44da-b038-4bf0e6243726","Type":"ContainerDied","Data":"e504fa1f460896f21aee1ae10674e54fa0dbeb205b8fbeaf1202c97b185bf4d5"}
Aug 03 23:33:29.388700 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:33:29.388684    2553 scope.go:117] "RemoveContainer" containerID="e504fa1f460896f21aee1ae10674e54fa0dbeb205b8fbeaf1202c97b185bf4d5"
Aug 03 23:33:30.387631 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:33:30.387592    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659b947d7b-744n4" event={"ID":"6f4d89e2-b214-44da-b038-4bf0e6243726","Type":"ContainerStarted","Data":"4b052cdbc5d5eb04881e91d09aaecba5a3ab67814eca2ed7f291097b0e560186"}
Aug 03 23:34:04.552552 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:04.552506    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:34:04.553079 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:04.552647    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 03 23:34:04.553079 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:04.552729    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs podName:5e420d03-b0de-4f89-936a-cdafff2847a9 nodeName:}" failed. No retries permitted until 2026-08-03 23:36:06.552693243 +0000 UTC m=+374.254399886 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs") pod "network-metrics-daemon-kld8m" (UID: "5e420d03-b0de-4f89-936a-cdafff2847a9") : secret "metrics-daemon-secret" not found
Aug 03 23:34:06.871883 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.871846    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"]
Aug 03 23:34:06.873536 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.873520    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:06.875941 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.875921    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Aug 03 23:34:06.877604 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.877587    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 03 23:34:06.878806 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.878787    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nt7pg\""
Aug 03 23:34:06.878916 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.878868    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 03 23:34:06.878916 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.878900    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Aug 03 23:34:06.885296 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.885273    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"]
Aug 03 23:34:06.941891 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.941869    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"]
Aug 03 23:34:06.943487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.943468    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"
Aug 03 23:34:06.944144 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.944125    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-zzb4f"]
Aug 03 23:34:06.945617 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.945600    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:06.946154 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.946135    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Aug 03 23:34:06.946251 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.946172    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-77tgw\""
Aug 03 23:34:06.946325 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.946281    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Aug 03 23:34:06.946325 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.946284    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-f79c65b86-kvn79"]
Aug 03 23:34:06.946538 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.946520    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Aug 03 23:34:06.946686 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.946661    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Aug 03 23:34:06.947958 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.947938    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:06.948047 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.947950    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4tr59\""
Aug 03 23:34:06.948399 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.948384    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Aug 03 23:34:06.948509 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.948496    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 03 23:34:06.948597 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.948578    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 03 23:34:06.948653 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.948591    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Aug 03 23:34:06.950677 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.950657    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 03 23:34:06.950784 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.950686    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Aug 03 23:34:06.951082 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.950925    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Aug 03 23:34:06.951082 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.950957    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kx6js\""
Aug 03 23:34:06.951082 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.950984    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 03 23:34:06.951082 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.950985    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Aug 03 23:34:06.951082 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.951052    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Aug 03 23:34:06.954307 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.954289    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"]
Aug 03 23:34:06.954400 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.954316    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Aug 03 23:34:06.958944 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.958924    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-zzb4f"]
Aug 03 23:34:06.961533 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.961513    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-f79c65b86-kvn79"]
Aug 03 23:34:06.967539 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.967520    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4n66\" (UniqueName: \"kubernetes.io/projected/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-kube-api-access-g4n66\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:06.967612 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.967574    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:06.967612 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:06.967595    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:07.068568 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.068546    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8888l\" (UniqueName: \"kubernetes.io/projected/7cc92f64-d9da-42ae-8883-c1e5dc28ceea-kube-api-access-8888l\") pod \"kube-storage-version-migrator-operator-7878f4b88c-pwwls\" (UID: \"7cc92f64-d9da-42ae-8883-c1e5dc28ceea\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"
Aug 03 23:34:07.068668 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.068587    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4n66\" (UniqueName: \"kubernetes.io/projected/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-kube-api-access-g4n66\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:07.068668 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.068606    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc92f64-d9da-42ae-8883-c1e5dc28ceea-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-pwwls\" (UID: \"7cc92f64-d9da-42ae-8883-c1e5dc28ceea\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"
Aug 03 23:34:07.068668 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.068622    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxxjm\" (UniqueName: \"kubernetes.io/projected/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-kube-api-access-nxxjm\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.068846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.068764    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-service-ca-bundle\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.068846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.068802    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-serving-cert\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.068952 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.068858    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-stats-auth\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.068952 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.068892    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7cc92f64-d9da-42ae-8883-c1e5dc28ceea-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-pwwls\" (UID: \"7cc92f64-d9da-42ae-8883-c1e5dc28ceea\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"
Aug 03 23:34:07.068952 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.068921    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.069103 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.068948    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-trusted-ca-bundle\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.069103 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.068972    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96rkr\" (UniqueName: \"kubernetes.io/projected/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-kube-api-access-96rkr\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.069103 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.069037    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.069103 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.069063    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-tmp\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.069103 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.069097    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:07.069345 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.069125    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:07.069345 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.069152    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-default-certificate\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.069345 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.069178    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-snapshots\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.069345 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:07.069259    2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 03 23:34:07.069345 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:07.069315    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls podName:a01ef950-7a61-40ba-a9c0-1d95c8b7b573 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:07.569300519 +0000 UTC m=+255.271007162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-csljn" (UID: "a01ef950-7a61-40ba-a9c0-1d95c8b7b573") : secret "cluster-monitoring-operator-tls" not found
Aug 03 23:34:07.069652 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.069636    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:07.077141 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.077123    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4n66\" (UniqueName: \"kubernetes.io/projected/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-kube-api-access-g4n66\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:07.170315 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170254    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-serving-cert\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.170315 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170290    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-stats-auth\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.170315 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170309    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7cc92f64-d9da-42ae-8883-c1e5dc28ceea-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-pwwls\" (UID: \"7cc92f64-d9da-42ae-8883-c1e5dc28ceea\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"
Aug 03 23:34:07.170540 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170326    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.170540 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170345    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-trusted-ca-bundle\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.170540 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170368    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96rkr\" (UniqueName: \"kubernetes.io/projected/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-kube-api-access-96rkr\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.170540 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170410    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.170540 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:07.170484    2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 03 23:34:07.170540 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:07.170536    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle podName:3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e nodeName:}" failed. No retries permitted until 2026-08-03 23:34:07.670514776 +0000 UTC m=+255.372221438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle") pod "router-default-f79c65b86-kvn79" (UID: "3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e") : configmap references non-existent config key: service-ca.crt
Aug 03 23:34:07.170893 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:07.170565    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs podName:3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e nodeName:}" failed. No retries permitted until 2026-08-03 23:34:07.670552774 +0000 UTC m=+255.372259418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs") pod "router-default-f79c65b86-kvn79" (UID: "3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e") : secret "router-metrics-certs-default" not found
Aug 03 23:34:07.170893 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170591    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-tmp\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.170893 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170659    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-default-certificate\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.170893 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170695    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-snapshots\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.170893 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170775    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8888l\" (UniqueName: \"kubernetes.io/projected/7cc92f64-d9da-42ae-8883-c1e5dc28ceea-kube-api-access-8888l\") pod \"kube-storage-version-migrator-operator-7878f4b88c-pwwls\" (UID: \"7cc92f64-d9da-42ae-8883-c1e5dc28ceea\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"
Aug 03 23:34:07.170893 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170832    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc92f64-d9da-42ae-8883-c1e5dc28ceea-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-pwwls\" (UID: \"7cc92f64-d9da-42ae-8883-c1e5dc28ceea\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"
Aug 03 23:34:07.170893 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170865    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxxjm\" (UniqueName: \"kubernetes.io/projected/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-kube-api-access-nxxjm\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.171393 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170913    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-service-ca-bundle\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.171393 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170985    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7cc92f64-d9da-42ae-8883-c1e5dc28ceea-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-pwwls\" (UID: \"7cc92f64-d9da-42ae-8883-c1e5dc28ceea\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"
Aug 03 23:34:07.171393 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.170992    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-tmp\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.171552 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.171520    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-service-ca-bundle\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.171607 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.171561    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-snapshots\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.171755 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.171738    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-trusted-ca-bundle\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.173050 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.173021    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-serving-cert\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.173145 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.173131    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-stats-auth\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.173194 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.173173    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc92f64-d9da-42ae-8883-c1e5dc28ceea-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-pwwls\" (UID: \"7cc92f64-d9da-42ae-8883-c1e5dc28ceea\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"
Aug 03 23:34:07.173641 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.173628    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-default-certificate\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.180517 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.180490    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96rkr\" (UniqueName: \"kubernetes.io/projected/2901f2fd-f8ce-47eb-a4c1-d0dba309961a-kube-api-access-96rkr\") pod \"insights-operator-77696685b-zzb4f\" (UID: \"2901f2fd-f8ce-47eb-a4c1-d0dba309961a\") " pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.181382 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.181363    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxxjm\" (UniqueName: \"kubernetes.io/projected/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-kube-api-access-nxxjm\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.181487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.181471    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8888l\" (UniqueName: \"kubernetes.io/projected/7cc92f64-d9da-42ae-8883-c1e5dc28ceea-kube-api-access-8888l\") pod \"kube-storage-version-migrator-operator-7878f4b88c-pwwls\" (UID: \"7cc92f64-d9da-42ae-8883-c1e5dc28ceea\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"
Aug 03 23:34:07.254470 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.254450    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"
Aug 03 23:34:07.261311 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.261290    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-zzb4f"
Aug 03 23:34:07.383595 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.383572    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls"]
Aug 03 23:34:07.386213 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:07.386187    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc92f64_d9da_42ae_8883_c1e5dc28ceea.slice/crio-2eaec8af72d58abfe64647b0684f1c27d36d38aabeb4d2263985ea8e30c9ea87 WatchSource:0}: Error finding container 2eaec8af72d58abfe64647b0684f1c27d36d38aabeb4d2263985ea8e30c9ea87: Status 404 returned error can't find the container with id 2eaec8af72d58abfe64647b0684f1c27d36d38aabeb4d2263985ea8e30c9ea87
Aug 03 23:34:07.400069 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.400047    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-zzb4f"]
Aug 03 23:34:07.402615 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:07.402591    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2901f2fd_f8ce_47eb_a4c1_d0dba309961a.slice/crio-5d423c9cdd615f0a07cf07c6a5ec0c0f746107576d4171f80b24572faf76de43 WatchSource:0}: Error finding container 5d423c9cdd615f0a07cf07c6a5ec0c0f746107576d4171f80b24572faf76de43: Status 404 returned error can't find the container with id 5d423c9cdd615f0a07cf07c6a5ec0c0f746107576d4171f80b24572faf76de43
Aug 03 23:34:07.475161 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.475135    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls" event={"ID":"7cc92f64-d9da-42ae-8883-c1e5dc28ceea","Type":"ContainerStarted","Data":"2eaec8af72d58abfe64647b0684f1c27d36d38aabeb4d2263985ea8e30c9ea87"}
Aug 03 23:34:07.476125 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.476104    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-zzb4f" event={"ID":"2901f2fd-f8ce-47eb-a4c1-d0dba309961a","Type":"ContainerStarted","Data":"5d423c9cdd615f0a07cf07c6a5ec0c0f746107576d4171f80b24572faf76de43"}
Aug 03 23:34:07.574334 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.574305    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:07.574463 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:07.574447    2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 03 23:34:07.574528 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:07.574519    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls podName:a01ef950-7a61-40ba-a9c0-1d95c8b7b573 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:08.574501856 +0000 UTC m=+256.276208501 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-csljn" (UID: "a01ef950-7a61-40ba-a9c0-1d95c8b7b573") : secret "cluster-monitoring-operator-tls" not found
Aug 03 23:34:07.675432 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.675397    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.675585 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:07.675443    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:07.675585 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:07.675536    2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 03 23:34:07.675585 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:07.675564    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle podName:3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e nodeName:}" failed. No retries permitted until 2026-08-03 23:34:08.675547813 +0000 UTC m=+256.377254456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle") pod "router-default-f79c65b86-kvn79" (UID: "3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e") : configmap references non-existent config key: service-ca.crt
Aug 03 23:34:07.675698 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:07.675590    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs podName:3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e nodeName:}" failed. No retries permitted until 2026-08-03 23:34:08.675582416 +0000 UTC m=+256.377289059 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs") pod "router-default-f79c65b86-kvn79" (UID: "3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e") : secret "router-metrics-certs-default" not found
Aug 03 23:34:08.584509 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:08.584469    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:08.584973 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:08.584619    2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 03 23:34:08.584973 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:08.584679    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls podName:a01ef950-7a61-40ba-a9c0-1d95c8b7b573 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:10.584663722 +0000 UTC m=+258.286370371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-csljn" (UID: "a01ef950-7a61-40ba-a9c0-1d95c8b7b573") : secret "cluster-monitoring-operator-tls" not found
Aug 03 23:34:08.685637 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:08.685590    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:08.685637 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:08.685645    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:08.685860 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:08.685796    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle podName:3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e nodeName:}" failed. No retries permitted until 2026-08-03 23:34:10.685772519 +0000 UTC m=+258.387479179 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle") pod "router-default-f79c65b86-kvn79" (UID: "3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e") : configmap references non-existent config key: service-ca.crt
Aug 03 23:34:08.685860 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:08.685808    2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 03 23:34:08.685860 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:08.685854    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs podName:3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e nodeName:}" failed. No retries permitted until 2026-08-03 23:34:10.685841153 +0000 UTC m=+258.387547795 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs") pod "router-default-f79c65b86-kvn79" (UID: "3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e") : secret "router-metrics-certs-default" not found
Aug 03 23:34:10.487058 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:10.487018    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls" event={"ID":"7cc92f64-d9da-42ae-8883-c1e5dc28ceea","Type":"ContainerStarted","Data":"3898a27dfe05396633432e99444b52c0c2003ca67d3ad57e2c30d85d6b665131"}
Aug 03 23:34:10.488689 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:10.488662    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-zzb4f" event={"ID":"2901f2fd-f8ce-47eb-a4c1-d0dba309961a","Type":"ContainerStarted","Data":"6fdf981a6849982b553aaadf6c3c29a14216e5f8e2d917e9f073361c96ef61fa"}
Aug 03 23:34:10.501104 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:10.501056    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls" podStartSLOduration=2.254998142 podStartE2EDuration="4.50103962s" totalImagesPullingTime="2.246041478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:34:10.501019973 +0000 UTC m=+258.202726637" watchObservedRunningTime="2026-08-03 23:34:10.50103962 +0000 UTC m=+258.202746286"
Aug 03 23:34:10.515830 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:10.515782    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-zzb4f" podStartSLOduration=2.287910422 podStartE2EDuration="4.515764201s" totalImagesPullingTime="2.227853779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:34:10.514845889 +0000 UTC m=+258.216552555" watchObservedRunningTime="2026-08-03 23:34:10.515764201 +0000 UTC m=+258.217470867"
Aug 03 23:34:10.601726 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:10.601672    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:10.602237 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:10.602109    2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 03 23:34:10.602237 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:10.602169    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls podName:a01ef950-7a61-40ba-a9c0-1d95c8b7b573 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:14.602149238 +0000 UTC m=+262.303855881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-csljn" (UID: "a01ef950-7a61-40ba-a9c0-1d95c8b7b573") : secret "cluster-monitoring-operator-tls" not found
Aug 03 23:34:10.702275 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:10.702233    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:10.702486 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:10.702302    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:10.702486 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:10.702405    2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 03 23:34:10.702486 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:10.702418    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle podName:3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e nodeName:}" failed. No retries permitted until 2026-08-03 23:34:14.702400859 +0000 UTC m=+262.404107504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle") pod "router-default-f79c65b86-kvn79" (UID: "3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e") : configmap references non-existent config key: service-ca.crt
Aug 03 23:34:10.702621 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:10.702494    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs podName:3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e nodeName:}" failed. No retries permitted until 2026-08-03 23:34:14.702478283 +0000 UTC m=+262.404184925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs") pod "router-default-f79c65b86-kvn79" (UID: "3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e") : secret "router-metrics-certs-default" not found
Aug 03 23:34:11.423325 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.423291    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tttx8"]
Aug 03 23:34:11.425527 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.425505    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.427950 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.427930    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 03 23:34:11.428071 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.427973    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lglgl\""
Aug 03 23:34:11.428071 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.427926    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 03 23:34:11.437044 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.437022    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tttx8"]
Aug 03 23:34:11.608361 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.608337    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-crio-socket\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.608688 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.608379    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-data-volume\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.608688 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.608442    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgc9w\" (UniqueName: \"kubernetes.io/projected/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-api-access-lgc9w\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.608688 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.608537    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.608688 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.608568    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.710079 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.710041    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-crio-socket\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.710206 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.710086    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-data-volume\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.710206 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.710114    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgc9w\" (UniqueName: \"kubernetes.io/projected/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-api-access-lgc9w\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.710206 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.710152    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.710206 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.710159    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-crio-socket\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.710206 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.710170    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.710382 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:11.710242    2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 03 23:34:11.710382 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:11.710309    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm podName:8573ff4f-bab3-4d29-b4d1-d8d8a0941687 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:12.210292341 +0000 UTC m=+259.911999002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tttx8" (UID: "8573ff4f-bab3-4d29-b4d1-d8d8a0941687") : configmap "kube-rbac-proxy" not found
Aug 03 23:34:11.710382 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:11.710244    2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 03 23:34:11.710382 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:11.710381    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls podName:8573ff4f-bab3-4d29-b4d1-d8d8a0941687 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:12.21036658 +0000 UTC m=+259.912073223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tttx8" (UID: "8573ff4f-bab3-4d29-b4d1-d8d8a0941687") : secret "insights-runtime-extractor-tls" not found
Aug 03 23:34:11.710558 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.710453    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-data-volume\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:11.722225 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:11.722197    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgc9w\" (UniqueName: \"kubernetes.io/projected/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-api-access-lgc9w\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:12.214214 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:12.214176    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:12.214214 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:12.214214    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:12.214417 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:12.214315    2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 03 23:34:12.214417 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:12.214328    2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 03 23:34:12.214417 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:12.214377    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm podName:8573ff4f-bab3-4d29-b4d1-d8d8a0941687 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:13.214363231 +0000 UTC m=+260.916069872 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tttx8" (UID: "8573ff4f-bab3-4d29-b4d1-d8d8a0941687") : configmap "kube-rbac-proxy" not found
Aug 03 23:34:12.214417 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:12.214390    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls podName:8573ff4f-bab3-4d29-b4d1-d8d8a0941687 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:13.214384296 +0000 UTC m=+260.916090938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tttx8" (UID: "8573ff4f-bab3-4d29-b4d1-d8d8a0941687") : secret "insights-runtime-extractor-tls" not found
Aug 03 23:34:13.222068 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:13.222037    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:13.222068 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:13.222068    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:13.222454 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:13.222145    2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 03 23:34:13.222454 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:13.222194    2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 03 23:34:13.222454 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:13.222207    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm podName:8573ff4f-bab3-4d29-b4d1-d8d8a0941687 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:15.222189093 +0000 UTC m=+262.923895748 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tttx8" (UID: "8573ff4f-bab3-4d29-b4d1-d8d8a0941687") : configmap "kube-rbac-proxy" not found
Aug 03 23:34:13.222454 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:13.222229    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls podName:8573ff4f-bab3-4d29-b4d1-d8d8a0941687 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:15.222218992 +0000 UTC m=+262.923925637 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tttx8" (UID: "8573ff4f-bab3-4d29-b4d1-d8d8a0941687") : secret "insights-runtime-extractor-tls" not found
Aug 03 23:34:14.389620 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:14.389596    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qm6s6_ced51f35-5c98-4126-a3b4-533675741b86/dns-node-resolver/0.log"
Aug 03 23:34:14.632493 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:14.632463    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:14.632640 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:14.632611    2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 03 23:34:14.632692 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:14.632678    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls podName:a01ef950-7a61-40ba-a9c0-1d95c8b7b573 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:22.632663225 +0000 UTC m=+270.334369867 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-csljn" (UID: "a01ef950-7a61-40ba-a9c0-1d95c8b7b573") : secret "cluster-monitoring-operator-tls" not found
Aug 03 23:34:14.733439 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:14.733409    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:14.733585 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:14.733449    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:14.733585 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:14.733538    2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 03 23:34:14.733585 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:14.733565    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle podName:3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e nodeName:}" failed. No retries permitted until 2026-08-03 23:34:22.733548059 +0000 UTC m=+270.435254701 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle") pod "router-default-f79c65b86-kvn79" (UID: "3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e") : configmap references non-existent config key: service-ca.crt
Aug 03 23:34:14.733585 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:14.733587    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs podName:3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e nodeName:}" failed. No retries permitted until 2026-08-03 23:34:22.73357815 +0000 UTC m=+270.435284792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs") pod "router-default-f79c65b86-kvn79" (UID: "3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e") : secret "router-metrics-certs-default" not found
Aug 03 23:34:15.236204 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:15.236173    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:15.236204 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:15.236206    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:15.236411 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:15.236289    2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 03 23:34:15.236411 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:15.236302    2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 03 23:34:15.236411 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:15.236352    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls podName:8573ff4f-bab3-4d29-b4d1-d8d8a0941687 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:19.236338069 +0000 UTC m=+266.938044712 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tttx8" (UID: "8573ff4f-bab3-4d29-b4d1-d8d8a0941687") : secret "insights-runtime-extractor-tls" not found
Aug 03 23:34:15.236411 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:15.236365    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm podName:8573ff4f-bab3-4d29-b4d1-d8d8a0941687 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:19.236357967 +0000 UTC m=+266.938064609 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tttx8" (UID: "8573ff4f-bab3-4d29-b4d1-d8d8a0941687") : configmap "kube-rbac-proxy" not found
Aug 03 23:34:15.389591 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:15.389563    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkhxt_79fe88bf-1fbb-4616-9dbb-3a3ccab24265/node-ca/0.log"
Aug 03 23:34:15.989448 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:15.989420    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-np8h4_966fe46a-d62f-4d39-ae52-5e9e12b681c6/serve-healthcheck-canary/0.log"
Aug 03 23:34:16.789864 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:16.789833    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-pwwls_7cc92f64-d9da-42ae-8883-c1e5dc28ceea/kube-storage-version-migrator-operator/0.log"
Aug 03 23:34:19.266628 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:19.266585    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:19.266628 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:19.266627    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:19.267111 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:19.266730    2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 03 23:34:19.267111 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:19.266792    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm podName:8573ff4f-bab3-4d29-b4d1-d8d8a0941687 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:27.266778374 +0000 UTC m=+274.968485017 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tttx8" (UID: "8573ff4f-bab3-4d29-b4d1-d8d8a0941687") : configmap "kube-rbac-proxy" not found
Aug 03 23:34:19.267111 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:19.266813    2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 03 23:34:19.267111 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:19.266860    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls podName:8573ff4f-bab3-4d29-b4d1-d8d8a0941687 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:27.266848088 +0000 UTC m=+274.968554731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tttx8" (UID: "8573ff4f-bab3-4d29-b4d1-d8d8a0941687") : secret "insights-runtime-extractor-tls" not found
Aug 03 23:34:22.693771 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:22.693733    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:22.696289 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:22.696268    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a01ef950-7a61-40ba-a9c0-1d95c8b7b573-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-csljn\" (UID: \"a01ef950-7a61-40ba-a9c0-1d95c8b7b573\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:22.782116 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:22.782080    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"
Aug 03 23:34:22.794590 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:22.794564    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:22.794699 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:22.794606    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:22.795346 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:22.795322    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-service-ca-bundle\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:22.797127 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:22.797109    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e-metrics-certs\") pod \"router-default-f79c65b86-kvn79\" (UID: \"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e\") " pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:22.867045 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:22.867015    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:22.903780 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:22.903655    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn"]
Aug 03 23:34:22.906082 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:22.906054    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda01ef950_7a61_40ba_a9c0_1d95c8b7b573.slice/crio-1c4bd6432d35d474d329a7b50ece4bf1423f09c3bc50e181e60901658503e2e3 WatchSource:0}: Error finding container 1c4bd6432d35d474d329a7b50ece4bf1423f09c3bc50e181e60901658503e2e3: Status 404 returned error can't find the container with id 1c4bd6432d35d474d329a7b50ece4bf1423f09c3bc50e181e60901658503e2e3
Aug 03 23:34:22.989679 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:22.989653    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-f79c65b86-kvn79"]
Aug 03 23:34:22.992376 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:22.992353    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5cf9aa_d5bb_4aa2_9c9a_6c3ac2f3c93e.slice/crio-4c3224c31bb03b3ed309d50b3618bd36f3071b3a04ac32c194e236d8e3396379 WatchSource:0}: Error finding container 4c3224c31bb03b3ed309d50b3618bd36f3071b3a04ac32c194e236d8e3396379: Status 404 returned error can't find the container with id 4c3224c31bb03b3ed309d50b3618bd36f3071b3a04ac32c194e236d8e3396379
Aug 03 23:34:23.531607 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:23.531558    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-f79c65b86-kvn79" event={"ID":"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e","Type":"ContainerStarted","Data":"6733aea7d62c998c80b3e70c740ad5875e78100493f4645d1824a7a5b9ff8ea7"}
Aug 03 23:34:23.531795 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:23.531616    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-f79c65b86-kvn79" event={"ID":"3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e","Type":"ContainerStarted","Data":"4c3224c31bb03b3ed309d50b3618bd36f3071b3a04ac32c194e236d8e3396379"}
Aug 03 23:34:23.532807 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:23.532772    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn" event={"ID":"a01ef950-7a61-40ba-a9c0-1d95c8b7b573","Type":"ContainerStarted","Data":"1c4bd6432d35d474d329a7b50ece4bf1423f09c3bc50e181e60901658503e2e3"}
Aug 03 23:34:23.553385 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:23.553338    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-f79c65b86-kvn79" podStartSLOduration=17.553319656 podStartE2EDuration="17.553319656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:34:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:34:23.552658445 +0000 UTC m=+271.254365109" watchObservedRunningTime="2026-08-03 23:34:23.553319656 +0000 UTC m=+271.255026323"
Aug 03 23:34:23.867850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:23.867769    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:23.870561 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:23.870539    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:24.536807 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:24.536742    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:24.538228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:24.538205    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-f79c65b86-kvn79"
Aug 03 23:34:25.541165 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:25.541127    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn" event={"ID":"a01ef950-7a61-40ba-a9c0-1d95c8b7b573","Type":"ContainerStarted","Data":"a6abbb775e2659d14cb3ab0d16c4207c98b1dcff31f1676b9cefa35edc269ef6"}
Aug 03 23:34:25.556363 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:25.556321    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-csljn" podStartSLOduration=17.720061639 podStartE2EDuration="19.556305913s" totalImagesPullingTime="1.836244274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:34:25.555581283 +0000 UTC m=+273.257287946" watchObservedRunningTime="2026-08-03 23:34:25.556305913 +0000 UTC m=+273.258012596"
Aug 03 23:34:27.329797 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:27.329766    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:27.329797 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:27.329801    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:27.330429 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:27.330404    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:27.332303 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:27.332278    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8573ff4f-bab3-4d29-b4d1-d8d8a0941687-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tttx8\" (UID: \"8573ff4f-bab3-4d29-b4d1-d8d8a0941687\") " pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:27.334215 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:27.334198    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tttx8"
Aug 03 23:34:27.456444 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:27.456416    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tttx8"]
Aug 03 23:34:27.459246 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:27.459219    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8573ff4f_bab3_4d29_b4d1_d8d8a0941687.slice/crio-f5e7e6dbebcd60dc7bb7dbbd9ccab0c378e6ced7e524423d594281243dfb324f WatchSource:0}: Error finding container f5e7e6dbebcd60dc7bb7dbbd9ccab0c378e6ced7e524423d594281243dfb324f: Status 404 returned error can't find the container with id f5e7e6dbebcd60dc7bb7dbbd9ccab0c378e6ced7e524423d594281243dfb324f
Aug 03 23:34:27.549630 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:27.549600    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tttx8" event={"ID":"8573ff4f-bab3-4d29-b4d1-d8d8a0941687","Type":"ContainerStarted","Data":"62007682f774dbca7e163c1e49084b56e2edeefc0f2f8327f6f9e8a78157d217"}
Aug 03 23:34:27.549765 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:27.549653    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tttx8" event={"ID":"8573ff4f-bab3-4d29-b4d1-d8d8a0941687","Type":"ContainerStarted","Data":"f5e7e6dbebcd60dc7bb7dbbd9ccab0c378e6ced7e524423d594281243dfb324f"}
Aug 03 23:34:28.554139 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:28.554104    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tttx8" event={"ID":"8573ff4f-bab3-4d29-b4d1-d8d8a0941687","Type":"ContainerStarted","Data":"7bf506b2cac765ae572e40c23578b3a80329806f5f93f8b91411752214d68d5e"}
Aug 03 23:34:30.564975 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:30.564939    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tttx8" event={"ID":"8573ff4f-bab3-4d29-b4d1-d8d8a0941687","Type":"ContainerStarted","Data":"b8c054519260eb8452069b03d3aebdeb1cb0f4838d94bb850d0d99574a754834"}
Aug 03 23:34:30.582184 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:30.582131    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tttx8" podStartSLOduration=17.350376264 podStartE2EDuration="19.582115766s" totalImagesPullingTime="2.231739502s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 23:34:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:34:30.580755286 +0000 UTC m=+278.282461947" watchObservedRunningTime="2026-08-03 23:34:30.582115766 +0000 UTC m=+278.283822429"
Aug 03 23:34:34.248115 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:34.248071    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6mrf2" podUID="731c0106-b5c5-4a64-957d-38254cee1862"
Aug 03 23:34:34.576780 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:34.576688    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:34:35.883667 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.883633    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq"]
Aug 03 23:34:35.885595 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.885580    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq"
Aug 03 23:34:35.887659 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.887637    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-dq5z2"]
Aug 03 23:34:35.888427 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.888399    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-ls6lg\""
Aug 03 23:34:35.888507 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.888433    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Aug 03 23:34:35.889770 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.889756    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-dq5z2"
Aug 03 23:34:35.892284 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.892266    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-cjv9n\""
Aug 03 23:34:35.892648 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.892557    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 03 23:34:35.893147 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.893130    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 03 23:34:35.897029 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.897010    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq"]
Aug 03 23:34:35.904318 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.904300    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5fd5d454d-sfdvz"]
Aug 03 23:34:35.904494 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:35.904478    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz" podUID="52d9b5aa-a776-405a-90a9-5341c61db6b2"
Aug 03 23:34:35.917433 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.917415    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-dq5z2"]
Aug 03 23:34:35.986575 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.986547    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/71ca6f7e-4eef-4260-8e66-1bfc9fd0de07-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-hqzdq\" (UID: \"71ca6f7e-4eef-4260-8e66-1bfc9fd0de07\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq"
Aug 03 23:34:35.986753 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:35.986598    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dksww\" (UniqueName: \"kubernetes.io/projected/9a63ea75-d9ca-4604-ad9d-a3e2fe4368d6-kube-api-access-dksww\") pod \"downloads-77c974ffcf-dq5z2\" (UID: \"9a63ea75-d9ca-4604-ad9d-a3e2fe4368d6\") " pod="openshift-console/downloads-77c974ffcf-dq5z2"
Aug 03 23:34:36.086936 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.086902    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dksww\" (UniqueName: \"kubernetes.io/projected/9a63ea75-d9ca-4604-ad9d-a3e2fe4368d6-kube-api-access-dksww\") pod \"downloads-77c974ffcf-dq5z2\" (UID: \"9a63ea75-d9ca-4604-ad9d-a3e2fe4368d6\") " pod="openshift-console/downloads-77c974ffcf-dq5z2"
Aug 03 23:34:36.087093 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.086974    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/71ca6f7e-4eef-4260-8e66-1bfc9fd0de07-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-hqzdq\" (UID: \"71ca6f7e-4eef-4260-8e66-1bfc9fd0de07\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq"
Aug 03 23:34:36.089513 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.089490    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/71ca6f7e-4eef-4260-8e66-1bfc9fd0de07-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-hqzdq\" (UID: \"71ca6f7e-4eef-4260-8e66-1bfc9fd0de07\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq"
Aug 03 23:34:36.096099 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.096071    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dksww\" (UniqueName: \"kubernetes.io/projected/9a63ea75-d9ca-4604-ad9d-a3e2fe4368d6-kube-api-access-dksww\") pod \"downloads-77c974ffcf-dq5z2\" (UID: \"9a63ea75-d9ca-4604-ad9d-a3e2fe4368d6\") " pod="openshift-console/downloads-77c974ffcf-dq5z2"
Aug 03 23:34:36.196206 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.196178    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq"
Aug 03 23:34:36.202013 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.201993    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-dq5z2"
Aug 03 23:34:36.332558 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.332501    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq"]
Aug 03 23:34:36.336937 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:36.336914    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ca6f7e_4eef_4260_8e66_1bfc9fd0de07.slice/crio-79146ab1f8578bde7bae0433cbffc3e16dc05eeb238c06fa7a3270a92bc56e9f WatchSource:0}: Error finding container 79146ab1f8578bde7bae0433cbffc3e16dc05eeb238c06fa7a3270a92bc56e9f: Status 404 returned error can't find the container with id 79146ab1f8578bde7bae0433cbffc3e16dc05eeb238c06fa7a3270a92bc56e9f
Aug 03 23:34:36.348356 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.348337    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-dq5z2"]
Aug 03 23:34:36.350409 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:36.350389    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a63ea75_d9ca_4604_ad9d_a3e2fe4368d6.slice/crio-7375ca416c6e49d3e04775f6d37ba574a302edfbc1614ee76cb6aee2747f8688 WatchSource:0}: Error finding container 7375ca416c6e49d3e04775f6d37ba574a302edfbc1614ee76cb6aee2747f8688: Status 404 returned error can't find the container with id 7375ca416c6e49d3e04775f6d37ba574a302edfbc1614ee76cb6aee2747f8688
Aug 03 23:34:36.588388 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.588305    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq" event={"ID":"71ca6f7e-4eef-4260-8e66-1bfc9fd0de07","Type":"ContainerStarted","Data":"79146ab1f8578bde7bae0433cbffc3e16dc05eeb238c06fa7a3270a92bc56e9f"}
Aug 03 23:34:36.589334 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.589307    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-dq5z2" event={"ID":"9a63ea75-d9ca-4604-ad9d-a3e2fe4368d6","Type":"ContainerStarted","Data":"7375ca416c6e49d3e04775f6d37ba574a302edfbc1614ee76cb6aee2747f8688"}
Aug 03 23:34:36.589334 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.589341    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:34:36.595538 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.595515    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:34:36.690994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.690941    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-installation-pull-secrets\") pod \"52d9b5aa-a776-405a-90a9-5341c61db6b2\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") "
Aug 03 23:34:36.690994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.690988    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-bound-sa-token\") pod \"52d9b5aa-a776-405a-90a9-5341c61db6b2\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") "
Aug 03 23:34:36.691184 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.691014    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5bg2\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-kube-api-access-m5bg2\") pod \"52d9b5aa-a776-405a-90a9-5341c61db6b2\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") "
Aug 03 23:34:36.691184 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.691041    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-image-registry-private-configuration\") pod \"52d9b5aa-a776-405a-90a9-5341c61db6b2\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") "
Aug 03 23:34:36.691184 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.691074    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-trusted-ca\") pod \"52d9b5aa-a776-405a-90a9-5341c61db6b2\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") "
Aug 03 23:34:36.691184 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.691089    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-certificates\") pod \"52d9b5aa-a776-405a-90a9-5341c61db6b2\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") "
Aug 03 23:34:36.691184 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.691124    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52d9b5aa-a776-405a-90a9-5341c61db6b2-ca-trust-extracted\") pod \"52d9b5aa-a776-405a-90a9-5341c61db6b2\" (UID: \"52d9b5aa-a776-405a-90a9-5341c61db6b2\") "
Aug 03 23:34:36.691515 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.691477    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "52d9b5aa-a776-405a-90a9-5341c61db6b2" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:34:36.691515 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.691492    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d9b5aa-a776-405a-90a9-5341c61db6b2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "52d9b5aa-a776-405a-90a9-5341c61db6b2" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 23:34:36.691660 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.691511    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "52d9b5aa-a776-405a-90a9-5341c61db6b2" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:34:36.693400 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.693373    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "52d9b5aa-a776-405a-90a9-5341c61db6b2" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:34:36.693496 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.693414    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-kube-api-access-m5bg2" (OuterVolumeSpecName: "kube-api-access-m5bg2") pod "52d9b5aa-a776-405a-90a9-5341c61db6b2" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2"). InnerVolumeSpecName "kube-api-access-m5bg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:34:36.693496 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.693453    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "52d9b5aa-a776-405a-90a9-5341c61db6b2" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:34:36.693566 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.693520    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "52d9b5aa-a776-405a-90a9-5341c61db6b2" (UID: "52d9b5aa-a776-405a-90a9-5341c61db6b2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:34:36.791870 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.791846    2553 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-bound-sa-token\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:34:36.791870 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.791866    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5bg2\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-kube-api-access-m5bg2\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:34:36.791999 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.791877    2553 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-image-registry-private-configuration\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:34:36.791999 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.791886    2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-trusted-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:34:36.791999 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.791894    2553 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-certificates\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:34:36.791999 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.791902    2553 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52d9b5aa-a776-405a-90a9-5341c61db6b2-ca-trust-extracted\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:34:36.791999 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:36.791910    2553 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52d9b5aa-a776-405a-90a9-5341c61db6b2-installation-pull-secrets\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:34:37.592472 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:37.592445    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5fd5d454d-sfdvz"
Aug 03 23:34:37.623201 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:37.623173    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5fd5d454d-sfdvz"]
Aug 03 23:34:37.629226 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:37.629200    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5fd5d454d-sfdvz"]
Aug 03 23:34:37.699374 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:37.699343    2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52d9b5aa-a776-405a-90a9-5341c61db6b2-registry-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:34:38.102019 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:38.101983    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:34:38.104644 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:38.104620    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/731c0106-b5c5-4a64-957d-38254cee1862-metrics-tls\") pod \"dns-default-6mrf2\" (UID: \"731c0106-b5c5-4a64-957d-38254cee1862\") " pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:34:38.179727 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:38.179688    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rmmw5\""
Aug 03 23:34:38.187772 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:38.187741    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:34:38.332231 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:38.332196    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6mrf2"]
Aug 03 23:34:38.335994 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:38.335967    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731c0106_b5c5_4a64_957d_38254cee1862.slice/crio-95dfe42ef7fd622b73b717fc2e247948719c13307c4cdf86f277b16d35930686 WatchSource:0}: Error finding container 95dfe42ef7fd622b73b717fc2e247948719c13307c4cdf86f277b16d35930686: Status 404 returned error can't find the container with id 95dfe42ef7fd622b73b717fc2e247948719c13307c4cdf86f277b16d35930686
Aug 03 23:34:38.597015 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:38.596972    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6mrf2" event={"ID":"731c0106-b5c5-4a64-957d-38254cee1862","Type":"ContainerStarted","Data":"95dfe42ef7fd622b73b717fc2e247948719c13307c4cdf86f277b16d35930686"}
Aug 03 23:34:38.598689 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:38.598597    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq" event={"ID":"71ca6f7e-4eef-4260-8e66-1bfc9fd0de07","Type":"ContainerStarted","Data":"b5856e47a41d6e4c46bf4d8ccb5f5b5a3ca2556ec6203a50fa02fd963025505b"}
Aug 03 23:34:38.598854 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:38.598842    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq"
Aug 03 23:34:38.604081 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:38.604061    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq"
Aug 03 23:34:38.614740 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:38.614665    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-hqzdq" podStartSLOduration=2.408861279 podStartE2EDuration="3.614643722s" totalImagesPullingTime="1.205782443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:34:38.61331152 +0000 UTC m=+286.315018209" watchObservedRunningTime="2026-08-03 23:34:38.614643722 +0000 UTC m=+286.316350387"
Aug 03 23:34:38.763784 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:38.763607    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52d9b5aa-a776-405a-90a9-5341c61db6b2" path="/var/lib/kubelet/pods/52d9b5aa-a776-405a-90a9-5341c61db6b2/volumes"
Aug 03 23:34:39.267275 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.267230    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"]
Aug 03 23:34:39.269761 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.269734    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.272255 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.272185    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Aug 03 23:34:39.273617 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.273594    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ngkkx\""
Aug 03 23:34:39.273889 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.273863    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 03 23:34:39.273889 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.273878    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Aug 03 23:34:39.280919 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.280888    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"]
Aug 03 23:34:39.314024 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.313963    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b64caa8-b147-47db-aba4-470d30ef0f48-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.314024 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.314020    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8jqq\" (UniqueName: \"kubernetes.io/projected/9b64caa8-b147-47db-aba4-470d30ef0f48-kube-api-access-r8jqq\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.314219 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.314084    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b64caa8-b147-47db-aba4-470d30ef0f48-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.314219 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.314164    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b64caa8-b147-47db-aba4-470d30ef0f48-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.415224 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.415191    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b64caa8-b147-47db-aba4-470d30ef0f48-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.415405 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.415279    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b64caa8-b147-47db-aba4-470d30ef0f48-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.415405 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.415322    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8jqq\" (UniqueName: \"kubernetes.io/projected/9b64caa8-b147-47db-aba4-470d30ef0f48-kube-api-access-r8jqq\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.415405 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.415384    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b64caa8-b147-47db-aba4-470d30ef0f48-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.415571 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:39.415423    2553 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Aug 03 23:34:39.415571 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:39.415495    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b64caa8-b147-47db-aba4-470d30ef0f48-prometheus-operator-tls podName:9b64caa8-b147-47db-aba4-470d30ef0f48 nodeName:}" failed. No retries permitted until 2026-08-03 23:34:39.915475813 +0000 UTC m=+287.617182469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/9b64caa8-b147-47db-aba4-470d30ef0f48-prometheus-operator-tls") pod "prometheus-operator-758d7cb874-4rfrq" (UID: "9b64caa8-b147-47db-aba4-470d30ef0f48") : secret "prometheus-operator-tls" not found
Aug 03 23:34:39.415990 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.415967    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b64caa8-b147-47db-aba4-470d30ef0f48-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.418244 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.418220    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b64caa8-b147-47db-aba4-470d30ef0f48-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.424736 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.424678    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8jqq\" (UniqueName: \"kubernetes.io/projected/9b64caa8-b147-47db-aba4-470d30ef0f48-kube-api-access-r8jqq\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.919941 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.919909    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b64caa8-b147-47db-aba4-470d30ef0f48-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:39.923284 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:39.923048    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b64caa8-b147-47db-aba4-470d30ef0f48-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-4rfrq\" (UID: \"9b64caa8-b147-47db-aba4-470d30ef0f48\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:40.183349 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:40.183315    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"
Aug 03 23:34:40.332784 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:40.332684    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-4rfrq"]
Aug 03 23:34:40.336919 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:40.336888    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b64caa8_b147_47db_aba4_470d30ef0f48.slice/crio-92f2ea9c913ac5dfd30ac7c12d95252d0463f571023abe5fde3d4c20e9bbb914 WatchSource:0}: Error finding container 92f2ea9c913ac5dfd30ac7c12d95252d0463f571023abe5fde3d4c20e9bbb914: Status 404 returned error can't find the container with id 92f2ea9c913ac5dfd30ac7c12d95252d0463f571023abe5fde3d4c20e9bbb914
Aug 03 23:34:40.608305 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:40.608214    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6mrf2" event={"ID":"731c0106-b5c5-4a64-957d-38254cee1862","Type":"ContainerStarted","Data":"c238a47aba08181c0a494a0b75543c3c46d3e7fd2fb118f728d4691eb5c84634"}
Aug 03 23:34:40.608305 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:40.608279    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6mrf2" event={"ID":"731c0106-b5c5-4a64-957d-38254cee1862","Type":"ContainerStarted","Data":"10587377e59b51db824610f41ff63d2a704d99f6c18b260ff917c7a968e3d7f9"}
Aug 03 23:34:40.608520 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:40.608325    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:34:40.609454 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:40.609431    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq" event={"ID":"9b64caa8-b147-47db-aba4-470d30ef0f48","Type":"ContainerStarted","Data":"92f2ea9c913ac5dfd30ac7c12d95252d0463f571023abe5fde3d4c20e9bbb914"}
Aug 03 23:34:40.624238 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:40.624194    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6mrf2" podStartSLOduration=251.016972951 podStartE2EDuration="4m12.624172702s" totalImagesPullingTime="1.607199751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:34:40.623634905 +0000 UTC m=+288.325341570" watchObservedRunningTime="2026-08-03 23:34:40.624172702 +0000 UTC m=+288.325879367"
Aug 03 23:34:41.197311 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.197185    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-656bd6c9fb-jbr4n"]
Aug 03 23:34:41.199520 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.199493    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.202052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.202026    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Aug 03 23:34:41.203209 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.203170    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fw459\""
Aug 03 23:34:41.203209 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.203192    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Aug 03 23:34:41.203462 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.203426    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Aug 03 23:34:41.203562 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.203505    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Aug 03 23:34:41.203562 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.203523    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Aug 03 23:34:41.208232 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.208177    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-656bd6c9fb-jbr4n"]
Aug 03 23:34:41.231940 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.231903    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-oauth-config\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.232075 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.231979    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-console-config\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.232075 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.232008    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c77wj\" (UniqueName: \"kubernetes.io/projected/39dc6b32-9e96-416c-90b6-7cee82af9363-kube-api-access-c77wj\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.232176 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.232086    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-oauth-serving-cert\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.232176 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.232126    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-serving-cert\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.232176 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.232156    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-service-ca\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.332993 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.332946    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-oauth-config\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.333191 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.333039    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-console-config\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.333191 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.333069    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c77wj\" (UniqueName: \"kubernetes.io/projected/39dc6b32-9e96-416c-90b6-7cee82af9363-kube-api-access-c77wj\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.333191 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.333106    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-oauth-serving-cert\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.333191 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.333138    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-serving-cert\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.333191 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.333164    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-service-ca\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.333972 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.333928    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-service-ca\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.334227 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.334179    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-console-config\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.334804 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.334777    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-oauth-serving-cert\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.336174 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.336151    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-oauth-config\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.336819 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.336796    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-serving-cert\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.341789 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.341749    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c77wj\" (UniqueName: \"kubernetes.io/projected/39dc6b32-9e96-416c-90b6-7cee82af9363-kube-api-access-c77wj\") pod \"console-656bd6c9fb-jbr4n\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") " pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.512811 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.512700    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:41.885467 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:41.885412    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-656bd6c9fb-jbr4n"]
Aug 03 23:34:42.620376 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:42.620289    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq" event={"ID":"9b64caa8-b147-47db-aba4-470d30ef0f48","Type":"ContainerStarted","Data":"42554ea9f27868c856747dec2ed021407f844c09da21bc4e3bf150b8ecb3e603"}
Aug 03 23:34:42.620376 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:42.620358    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq" event={"ID":"9b64caa8-b147-47db-aba4-470d30ef0f48","Type":"ContainerStarted","Data":"0a3689a7e0f06c277fd528e235756a765d76d1ee775d8f993e4cfe6fde8f1919"}
Aug 03 23:34:42.621992 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:42.621938    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656bd6c9fb-jbr4n" event={"ID":"39dc6b32-9e96-416c-90b6-7cee82af9363","Type":"ContainerStarted","Data":"8b2995beb791d1fab447c0f645bf7fff990784369ccc4c393b70dd9ce9ee8788"}
Aug 03 23:34:42.638823 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:42.638766    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-4rfrq" podStartSLOduration=2.191696761 podStartE2EDuration="3.638745718s" totalImagesPullingTime="1.447048957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:34:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:34:42.636794744 +0000 UTC m=+290.338501409" watchObservedRunningTime="2026-08-03 23:34:42.638745718 +0000 UTC m=+290.340452383"
Aug 03 23:34:44.650447 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.650408    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wmvr5"]
Aug 03 23:34:44.655789 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.655763    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.658241 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.658097    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 03 23:34:44.658241 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.658111    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zn6kg\""
Aug 03 23:34:44.658440 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.658414    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 03 23:34:44.658497 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.658480    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 03 23:34:44.762481 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.762387    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cadbad4c-dec8-4dfd-959c-3570a1dd486f-sys\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.762481 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.762444    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cadbad4c-dec8-4dfd-959c-3570a1dd486f-metrics-client-ca\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.762748 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.762490    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.762748 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.762514    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gclhc\" (UniqueName: \"kubernetes.io/projected/cadbad4c-dec8-4dfd-959c-3570a1dd486f-kube-api-access-gclhc\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.762748 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.762537    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-textfile\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.762748 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.762570    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-wtmp\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.762960 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.762753    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cadbad4c-dec8-4dfd-959c-3570a1dd486f-root\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.762960 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.762798    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-accelerators-collector-config\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.762960 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.762827    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-tls\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.863758    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cadbad4c-dec8-4dfd-959c-3570a1dd486f-root\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.863809    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-accelerators-collector-config\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.863848    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-tls\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.863882    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cadbad4c-dec8-4dfd-959c-3570a1dd486f-sys\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.863913    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cadbad4c-dec8-4dfd-959c-3570a1dd486f-metrics-client-ca\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.863937    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.863960    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gclhc\" (UniqueName: \"kubernetes.io/projected/cadbad4c-dec8-4dfd-959c-3570a1dd486f-kube-api-access-gclhc\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.863986    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-textfile\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.864024    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-wtmp\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.864091    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cadbad4c-dec8-4dfd-959c-3570a1dd486f-sys\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.864147    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cadbad4c-dec8-4dfd-959c-3570a1dd486f-root\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.864202    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-wtmp\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.864730    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cadbad4c-dec8-4dfd-959c-3570a1dd486f-metrics-client-ca\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:44.864839    2553 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Aug 03 23:34:44.864994 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:44.864891    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-tls podName:cadbad4c-dec8-4dfd-959c-3570a1dd486f nodeName:}" failed. No retries permitted until 2026-08-03 23:34:45.364874619 +0000 UTC m=+293.066581261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-tls") pod "node-exporter-wmvr5" (UID: "cadbad4c-dec8-4dfd-959c-3570a1dd486f") : secret "node-exporter-tls" not found
Aug 03 23:34:44.865852 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.865051    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-textfile\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.865852 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.865094    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-accelerators-collector-config\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.868040 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.868012    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:44.876211 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:44.876167    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gclhc\" (UniqueName: \"kubernetes.io/projected/cadbad4c-dec8-4dfd-959c-3570a1dd486f-kube-api-access-gclhc\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:45.368238 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.368199    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-tls\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:45.370964 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.370936    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cadbad4c-dec8-4dfd-959c-3570a1dd486f-node-exporter-tls\") pod \"node-exporter-wmvr5\" (UID: \"cadbad4c-dec8-4dfd-959c-3570a1dd486f\") " pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:45.569384 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.569344    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wmvr5"
Aug 03 23:34:45.582210 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:45.582179    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcadbad4c_dec8_4dfd_959c_3570a1dd486f.slice/crio-f2f71cfd767f24fb0f4e49659aa8af4cfdb6027e3fabb2ebced4dbbb556475e2 WatchSource:0}: Error finding container f2f71cfd767f24fb0f4e49659aa8af4cfdb6027e3fabb2ebced4dbbb556475e2: Status 404 returned error can't find the container with id f2f71cfd767f24fb0f4e49659aa8af4cfdb6027e3fabb2ebced4dbbb556475e2
Aug 03 23:34:45.637362 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.637326    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656bd6c9fb-jbr4n" event={"ID":"39dc6b32-9e96-416c-90b6-7cee82af9363","Type":"ContainerStarted","Data":"65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef"}
Aug 03 23:34:45.638903 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.638865    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wmvr5" event={"ID":"cadbad4c-dec8-4dfd-959c-3570a1dd486f","Type":"ContainerStarted","Data":"f2f71cfd767f24fb0f4e49659aa8af4cfdb6027e3fabb2ebced4dbbb556475e2"}
Aug 03 23:34:45.659130 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.659080    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-656bd6c9fb-jbr4n" podStartSLOduration=1.40899389 podStartE2EDuration="4.659062715s" totalImagesPullingTime="3.250068825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:34:45.658064749 +0000 UTC m=+293.359771413" watchObservedRunningTime="2026-08-03 23:34:45.659062715 +0000 UTC m=+293.360769379"
Aug 03 23:34:45.694153 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.694126    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 03 23:34:45.698033 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.698013    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.700578 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.700558    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Aug 03 23:34:45.700779 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.700623    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Aug 03 23:34:45.700889 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.700846    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Aug 03 23:34:45.700954 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.700894    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cw6bw\""
Aug 03 23:34:45.701017 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.700892    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Aug 03 23:34:45.701076 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.701029    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Aug 03 23:34:45.701446 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.701208    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Aug 03 23:34:45.701446 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.701253    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Aug 03 23:34:45.701446 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.701287    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Aug 03 23:34:45.701725 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.701691    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Aug 03 23:34:45.716632 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.716612    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 03 23:34:45.772618 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.772588    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-config-out\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.772813 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.772627    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvr8z\" (UniqueName: \"kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-kube-api-access-wvr8z\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.772813 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.772661    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.772813 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.772793    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.772981 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.772837    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.772981 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.772867    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-config-volume\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.772981 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.772899    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-web-config\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.773142 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.772999    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.773142 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.773038    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.773142 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.773064    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.773142 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.773093    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.773142 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.773132    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-tls-assets\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.773362 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.773162    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.874730 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874556    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-tls-assets\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.874730 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874603    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.874730 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874640    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-config-out\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.874730 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874664    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvr8z\" (UniqueName: \"kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-kube-api-access-wvr8z\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.874730 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874695    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.875078 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874776    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.875078 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874805    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.875078 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874832    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-config-volume\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.875078 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874861    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-web-config\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.875078 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874911    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.875078 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874939    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.875078 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.874966    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.875078 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.875009    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.875915 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.875787    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.879030 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.877718    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.879030 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.878665    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.880554 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.880507    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-tls-assets\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.880782 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.880756    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.881738 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.881692    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-config-out\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.881850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.881700    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.882022 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.881975    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-config-volume\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.882126 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.882085    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.883137 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.882977    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.883375 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.883347    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.884649 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.884628    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvr8z\" (UniqueName: \"kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-kube-api-access-wvr8z\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:45.884867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:45.884850    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-web-config\") pod \"alertmanager-main-0\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:46.012456 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:46.012403    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:34:46.181746 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:46.181692    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 03 23:34:48.758955 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:34:48.758915    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-kld8m" podUID="5e420d03-b0de-4f89-936a-cdafff2847a9"
Aug 03 23:34:50.617386 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.617336    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6mrf2"
Aug 03 23:34:50.835525 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.834159    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 23:34:50.838307 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.838281    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.840783 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.840741    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Aug 03 23:34:50.840984 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.840966    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Aug 03 23:34:50.841448 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.841429    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-297s8milb60dv\""
Aug 03 23:34:50.841876 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.841793    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Aug 03 23:34:50.841876 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.841845    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Aug 03 23:34:50.841876 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.841849    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Aug 03 23:34:50.842335 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.842198    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Aug 03 23:34:50.842525 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.842504    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Aug 03 23:34:50.842873 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.842853    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Aug 03 23:34:50.843028 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.843007    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mdpdv\""
Aug 03 23:34:50.843139 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.843097    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Aug 03 23:34:50.843253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.843222    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Aug 03 23:34:50.843253 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.843008    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Aug 03 23:34:50.843360 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.843347    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Aug 03 23:34:50.844615 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.844594    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Aug 03 23:34:50.848688 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.848651    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Aug 03 23:34:50.849990 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.849949    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Aug 03 23:34:50.856876 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.856855    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 23:34:50.923134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923053    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923134 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923098    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923360 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923190    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923360 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923248    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923360 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923277    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923360 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923303    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923360 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923354    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923618 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923382    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923618 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923488    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923618 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923531    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923618 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923556    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923653    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923684    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923737    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923770    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923794    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923807    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfqsm\" (UniqueName: \"kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-kube-api-access-sfqsm\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.923850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923845    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.924197 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923889    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:50.924197 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:50.923933    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.025905 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.025821    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.025905 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.025887    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026239 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.025925    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026239 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.025954    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026239 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.025989    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026239 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026012    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026239 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026043    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026239 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026072    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026239 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026097    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026239 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026160    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026239 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026182    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026863 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026308    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026863 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026358    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026863 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026401    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026863 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026428    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfqsm\" (UniqueName: \"kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-kube-api-access-sfqsm\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026863 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026459    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026863 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026494    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026863 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026528    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026863 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026566    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.026863 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026598    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.027371 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.026927    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.027371 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.027034    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.027566 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.027503    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.028156 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.027859    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.029211 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.028902    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.030093 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.030073    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.035785 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.030757    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.035912 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.031230    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.035995 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.033605    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.036079 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.034073    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.036172 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.034635    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.036261 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.034667    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.036351 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.035592    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.036433 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.035737    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.036524 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.036108    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.037181 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.037142    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.037876 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.037610    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.037876 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.037623    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.038212 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.038171    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.040426 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.040403    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfqsm\" (UniqueName: \"kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-kube-api-access-sfqsm\") pod \"prometheus-k8s-0\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.151875 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.151793    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:34:51.513775 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.513741    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:51.513941 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.513818    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:51.519222 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.519195    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:51.667699 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:51.667670    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:34:53.469974 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:53.469941    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c26563_c974_4779_b237_4412c5231119.slice/crio-cb8f9545018acc59a6473093b46ee26d3315fe15b0851f1083c5d3d0d85f43e0 WatchSource:0}: Error finding container cb8f9545018acc59a6473093b46ee26d3315fe15b0851f1083c5d3d0d85f43e0: Status 404 returned error can't find the container with id cb8f9545018acc59a6473093b46ee26d3315fe15b0851f1083c5d3d0d85f43e0
Aug 03 23:34:53.670880 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:53.670699    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerStarted","Data":"cb8f9545018acc59a6473093b46ee26d3315fe15b0851f1083c5d3d0d85f43e0"}
Aug 03 23:34:53.757667 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:53.757641    2553 kubelet.go:1648] "Image garbage collection succeeded"
Aug 03 23:34:53.879611 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:53.879580    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 23:34:53.880914 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:34:53.880880    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5c1c722_24ba_4493_8fc6_80b58b5ca7b8.slice/crio-7f7f82eba45761075d834ff6b3ad71f96436fe160be6d2509b773bd3d7235df0 WatchSource:0}: Error finding container 7f7f82eba45761075d834ff6b3ad71f96436fe160be6d2509b773bd3d7235df0: Status 404 returned error can't find the container with id 7f7f82eba45761075d834ff6b3ad71f96436fe160be6d2509b773bd3d7235df0
Aug 03 23:34:53.883095 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:53.883077    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 23:34:54.679386 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:54.678732    2553 generic.go:356] "Generic (PLEG): container finished" podID="cadbad4c-dec8-4dfd-959c-3570a1dd486f" containerID="d5dfe0bfb7f8767984244d7211a0cf0f687cfaa4deae2e094dfe4c91ebf05aef" exitCode=0
Aug 03 23:34:54.679386 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:54.678841    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wmvr5" event={"ID":"cadbad4c-dec8-4dfd-959c-3570a1dd486f","Type":"ContainerDied","Data":"d5dfe0bfb7f8767984244d7211a0cf0f687cfaa4deae2e094dfe4c91ebf05aef"}
Aug 03 23:34:54.681635 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:54.681596    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerStarted","Data":"7f7f82eba45761075d834ff6b3ad71f96436fe160be6d2509b773bd3d7235df0"}
Aug 03 23:34:54.684490 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:54.684198    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-dq5z2" event={"ID":"9a63ea75-d9ca-4604-ad9d-a3e2fe4368d6","Type":"ContainerStarted","Data":"1eb80600c2a800d637bdb55017ff5ff7f8d1bb866ee7fb475d8566be49700efa"}
Aug 03 23:34:54.684675 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:54.684633    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-dq5z2"
Aug 03 23:34:54.701524 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:54.701502    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-dq5z2"
Aug 03 23:34:54.715389 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:54.714902    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-dq5z2" podStartSLOduration=2.289898049 podStartE2EDuration="19.71488172s" totalImagesPullingTime="17.424983671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:34:54.714263593 +0000 UTC m=+302.415970254" watchObservedRunningTime="2026-08-03 23:34:54.71488172 +0000 UTC m=+302.416588385"
Aug 03 23:34:55.692182 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:55.692144    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wmvr5" event={"ID":"cadbad4c-dec8-4dfd-959c-3570a1dd486f","Type":"ContainerStarted","Data":"2e9369c4a82a4218837cb770c63592986e199284cf9bf91c91abec06876602ce"}
Aug 03 23:34:55.692633 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:55.692219    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wmvr5" event={"ID":"cadbad4c-dec8-4dfd-959c-3570a1dd486f","Type":"ContainerStarted","Data":"838aeae82704b5ab77f057c80892944a2bb4bb687cfde67f14137040c1b02a94"}
Aug 03 23:34:55.712254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:55.712196    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wmvr5" podStartSLOduration=3.492274722 podStartE2EDuration="11.712176505s" totalImagesPullingTime="8.144755082s" totalInitContainerRuntime="75.146701ms" isStatefulPod=true podCreationTimestamp="2026-08-03 23:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:34:55.710065844 +0000 UTC m=+303.411772511" watchObservedRunningTime="2026-08-03 23:34:55.712176505 +0000 UTC m=+303.413883170"
Aug 03 23:34:56.698344 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:56.698305    2553 generic.go:356] "Generic (PLEG): container finished" podID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerID="ca4f0cf080dd565459af099d663cd94679b573b519ca171023e0673cdb80e71e" exitCode=0
Aug 03 23:34:56.698801 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:56.698402    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerDied","Data":"ca4f0cf080dd565459af099d663cd94679b573b519ca171023e0673cdb80e71e"}
Aug 03 23:34:56.700576 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:56.700552    2553 generic.go:356] "Generic (PLEG): container finished" podID="70c26563-c974-4779-b237-4412c5231119" containerID="1b77e7a2757d6edb7dabb56dfe9c5a232798410e96d77d96ca3628f62ea8f451" exitCode=0
Aug 03 23:34:56.700668 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:56.700637    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerDied","Data":"1b77e7a2757d6edb7dabb56dfe9c5a232798410e96d77d96ca3628f62ea8f451"}
Aug 03 23:34:59.718469 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:59.718434    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerStarted","Data":"7614c4e711a9b1c332e260a5d6c096623e9b9fc52fa605d8a0439d1d08ba2481"}
Aug 03 23:34:59.718469 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:34:59.718524    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerStarted","Data":"0045283d9a71416e348411496b393ea8648710647edbef000622ee9aad3dd400"}
Aug 03 23:35:01.731525 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:01.731487    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerStarted","Data":"2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d"}
Aug 03 23:35:01.732047 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:01.731575    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerStarted","Data":"16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863"}
Aug 03 23:35:01.735013 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:01.734848    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerStarted","Data":"8621c8ec797fc7466a0b0d4c5c75d7b0a1e58eb1715d1e81718bce15a255709a"}
Aug 03 23:35:01.735013 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:01.734922    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerStarted","Data":"1ce3c9dc4970094bf6d2fae8f30a7c0e23cd9d2cd6c5eb3bf8e44510bb4257b6"}
Aug 03 23:35:01.735013 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:01.734959    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerStarted","Data":"d943040d15613371c2e2406503f192b5a7a3e2275b3c08bb88bc558d6391a181"}
Aug 03 23:35:02.479634 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:02.478858    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-656bd6c9fb-jbr4n"]
Aug 03 23:35:02.760255 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:02.759934    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:35:03.746155 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:03.746128    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerStarted","Data":"06ec45803a47e2af067c83d707cb7b256ff4cfa3876cc80de297659dc01ac1f4"}
Aug 03 23:35:04.753063 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:04.753029    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerStarted","Data":"b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6"}
Aug 03 23:35:04.753443 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:04.753093    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerStarted","Data":"79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b"}
Aug 03 23:35:04.753443 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:04.753120    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerStarted","Data":"a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032"}
Aug 03 23:35:04.753443 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:04.753145    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerStarted","Data":"9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32"}
Aug 03 23:35:04.782897 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:04.782856    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.639757817 podStartE2EDuration="14.782840018s" totalImagesPullingTime="9.120592223s" totalInitContainerRuntime="22.489978ms" isStatefulPod=false podCreationTimestamp="2026-08-03 23:34:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:35:04.780541172 +0000 UTC m=+312.482247851" watchObservedRunningTime="2026-08-03 23:35:04.782840018 +0000 UTC m=+312.484546686"
Aug 03 23:35:04.805284 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:04.805246    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=13.14524984 podStartE2EDuration="19.805234068s" totalImagesPullingTime="6.630746764s" totalInitContainerRuntime="29.237464ms" isStatefulPod=false podCreationTimestamp="2026-08-03 23:34:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:35:04.803120318 +0000 UTC m=+312.504826981" watchObservedRunningTime="2026-08-03 23:35:04.805234068 +0000 UTC m=+312.506940733"
Aug 03 23:35:06.152015 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:06.151979    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:35:27.506529 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.506489    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-656bd6c9fb-jbr4n" podUID="39dc6b32-9e96-416c-90b6-7cee82af9363" containerName="console" containerID="cri-o://65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef" gracePeriod=15
Aug 03 23:35:27.790768 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.790745    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:35:27.846875 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.846851    2553 generic.go:356] "Generic (PLEG): container finished" podID="39dc6b32-9e96-416c-90b6-7cee82af9363" containerID="65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef" exitCode=0
Aug 03 23:35:27.847006 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.846891    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656bd6c9fb-jbr4n"
Aug 03 23:35:27.847006 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.846897    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656bd6c9fb-jbr4n" event={"ID":"39dc6b32-9e96-416c-90b6-7cee82af9363","Type":"ContainerDied","Data":"65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef"}
Aug 03 23:35:27.847006 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.846940    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656bd6c9fb-jbr4n" event={"ID":"39dc6b32-9e96-416c-90b6-7cee82af9363","Type":"ContainerDied","Data":"8b2995beb791d1fab447c0f645bf7fff990784369ccc4c393b70dd9ce9ee8788"}
Aug 03 23:35:27.847006 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.846971    2553 scope.go:117] "RemoveContainer" containerID="65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef"
Aug 03 23:35:27.854977 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.854960    2553 scope.go:117] "RemoveContainer" containerID="65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef"
Aug 03 23:35:27.855229 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:35:27.855209    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef\": container with ID starting with 65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef not found: ID does not exist" containerID="65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef"
Aug 03 23:35:27.855295 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.855238    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef"} err="failed to get container status \"65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef\": rpc error: code = NotFound desc = could not find container \"65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef\": container with ID starting with 65c4c4da8f305be73b7251ac246bd0abd9d1cbe50c6d4970e814eb54c81d8aef not found: ID does not exist"
Aug 03 23:35:27.867514 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.867495    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-oauth-config\") pod \"39dc6b32-9e96-416c-90b6-7cee82af9363\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") "
Aug 03 23:35:27.867579 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.867523    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-console-config\") pod \"39dc6b32-9e96-416c-90b6-7cee82af9363\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") "
Aug 03 23:35:27.867579 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.867556    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-service-ca\") pod \"39dc6b32-9e96-416c-90b6-7cee82af9363\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") "
Aug 03 23:35:27.867675 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.867610    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-serving-cert\") pod \"39dc6b32-9e96-416c-90b6-7cee82af9363\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") "
Aug 03 23:35:27.867755 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.867684    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-oauth-serving-cert\") pod \"39dc6b32-9e96-416c-90b6-7cee82af9363\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") "
Aug 03 23:35:27.867755 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.867741    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c77wj\" (UniqueName: \"kubernetes.io/projected/39dc6b32-9e96-416c-90b6-7cee82af9363-kube-api-access-c77wj\") pod \"39dc6b32-9e96-416c-90b6-7cee82af9363\" (UID: \"39dc6b32-9e96-416c-90b6-7cee82af9363\") "
Aug 03 23:35:27.868015 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.867892    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-console-config" (OuterVolumeSpecName: "console-config") pod "39dc6b32-9e96-416c-90b6-7cee82af9363" (UID: "39dc6b32-9e96-416c-90b6-7cee82af9363"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:35:27.868116 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.868009    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-service-ca" (OuterVolumeSpecName: "service-ca") pod "39dc6b32-9e96-416c-90b6-7cee82af9363" (UID: "39dc6b32-9e96-416c-90b6-7cee82af9363"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:35:27.868116 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.868038    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "39dc6b32-9e96-416c-90b6-7cee82af9363" (UID: "39dc6b32-9e96-416c-90b6-7cee82af9363"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:35:27.869863 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.869831    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "39dc6b32-9e96-416c-90b6-7cee82af9363" (UID: "39dc6b32-9e96-416c-90b6-7cee82af9363"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:35:27.869863 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.869845    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "39dc6b32-9e96-416c-90b6-7cee82af9363" (UID: "39dc6b32-9e96-416c-90b6-7cee82af9363"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:35:27.869989 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.869971    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39dc6b32-9e96-416c-90b6-7cee82af9363-kube-api-access-c77wj" (OuterVolumeSpecName: "kube-api-access-c77wj") pod "39dc6b32-9e96-416c-90b6-7cee82af9363" (UID: "39dc6b32-9e96-416c-90b6-7cee82af9363"). InnerVolumeSpecName "kube-api-access-c77wj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:35:27.969326 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.969302    2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-oauth-serving-cert\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:35:27.969326 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.969322    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c77wj\" (UniqueName: \"kubernetes.io/projected/39dc6b32-9e96-416c-90b6-7cee82af9363-kube-api-access-c77wj\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:35:27.969462 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.969333    2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-oauth-config\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:35:27.969462 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.969342    2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-console-config\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:35:27.969462 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.969350    2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39dc6b32-9e96-416c-90b6-7cee82af9363-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:35:27.969462 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:27.969358    2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39dc6b32-9e96-416c-90b6-7cee82af9363-console-serving-cert\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:35:28.167897 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:28.167870    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-656bd6c9fb-jbr4n"]
Aug 03 23:35:28.172139 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:28.172119    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-656bd6c9fb-jbr4n"]
Aug 03 23:35:28.763033 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:28.763000    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39dc6b32-9e96-416c-90b6-7cee82af9363" path="/var/lib/kubelet/pods/39dc6b32-9e96-416c-90b6-7cee82af9363/volumes"
Aug 03 23:35:40.895108 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:40.895075    2553 generic.go:356] "Generic (PLEG): container finished" podID="7cc92f64-d9da-42ae-8883-c1e5dc28ceea" containerID="3898a27dfe05396633432e99444b52c0c2003ca67d3ad57e2c30d85d6b665131" exitCode=0
Aug 03 23:35:40.895545 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:40.895145    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls" event={"ID":"7cc92f64-d9da-42ae-8883-c1e5dc28ceea","Type":"ContainerDied","Data":"3898a27dfe05396633432e99444b52c0c2003ca67d3ad57e2c30d85d6b665131"}
Aug 03 23:35:40.895545 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:40.895465    2553 scope.go:117] "RemoveContainer" containerID="3898a27dfe05396633432e99444b52c0c2003ca67d3ad57e2c30d85d6b665131"
Aug 03 23:35:40.896699 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:40.896679    2553 generic.go:356] "Generic (PLEG): container finished" podID="2901f2fd-f8ce-47eb-a4c1-d0dba309961a" containerID="6fdf981a6849982b553aaadf6c3c29a14216e5f8e2d917e9f073361c96ef61fa" exitCode=0
Aug 03 23:35:40.896823 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:40.896734    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-zzb4f" event={"ID":"2901f2fd-f8ce-47eb-a4c1-d0dba309961a","Type":"ContainerDied","Data":"6fdf981a6849982b553aaadf6c3c29a14216e5f8e2d917e9f073361c96ef61fa"}
Aug 03 23:35:40.896959 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:40.896948    2553 scope.go:117] "RemoveContainer" containerID="6fdf981a6849982b553aaadf6c3c29a14216e5f8e2d917e9f073361c96ef61fa"
Aug 03 23:35:41.902453 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:41.902421    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-pwwls" event={"ID":"7cc92f64-d9da-42ae-8883-c1e5dc28ceea","Type":"ContainerStarted","Data":"d09272b93b9981a1a38d409609f0f8a587ecc876ef07d94f5f391eeb7419f3ec"}
Aug 03 23:35:41.904157 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:41.904134    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-zzb4f" event={"ID":"2901f2fd-f8ce-47eb-a4c1-d0dba309961a","Type":"ContainerStarted","Data":"54d9d62edbc082afcff0f4390ee027e9805cb969a0ae6e1b6fb984bd0ade5b93"}
Aug 03 23:35:51.152770 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:51.152675    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:35:51.169030 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:51.169004    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:35:51.959471 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:35:51.959441    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:04.858430 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:04.858398    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 03 23:36:04.858923 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:04.858666    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="alertmanager" containerID="cri-o://0045283d9a71416e348411496b393ea8648710647edbef000622ee9aad3dd400" gracePeriod=120
Aug 03 23:36:04.858923 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:04.858760    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy-web" containerID="cri-o://d943040d15613371c2e2406503f192b5a7a3e2275b3c08bb88bc558d6391a181" gracePeriod=120
Aug 03 23:36:04.858923 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:04.858782    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="config-reloader" containerID="cri-o://7614c4e711a9b1c332e260a5d6c096623e9b9fc52fa605d8a0439d1d08ba2481" gracePeriod=120
Aug 03 23:36:04.858923 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:04.858824    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="prom-label-proxy" containerID="cri-o://06ec45803a47e2af067c83d707cb7b256ff4cfa3876cc80de297659dc01ac1f4" gracePeriod=120
Aug 03 23:36:04.858923 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:04.858758    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy-metric" containerID="cri-o://8621c8ec797fc7466a0b0d4c5c75d7b0a1e58eb1715d1e81718bce15a255709a" gracePeriod=120
Aug 03 23:36:04.858923 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:04.858774    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy" containerID="cri-o://1ce3c9dc4970094bf6d2fae8f30a7c0e23cd9d2cd6c5eb3bf8e44510bb4257b6" gracePeriod=120
Aug 03 23:36:05.001381 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:05.001355    2553 generic.go:356] "Generic (PLEG): container finished" podID="70c26563-c974-4779-b237-4412c5231119" containerID="06ec45803a47e2af067c83d707cb7b256ff4cfa3876cc80de297659dc01ac1f4" exitCode=0
Aug 03 23:36:05.001381 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:05.001377    2553 generic.go:356] "Generic (PLEG): container finished" podID="70c26563-c974-4779-b237-4412c5231119" containerID="1ce3c9dc4970094bf6d2fae8f30a7c0e23cd9d2cd6c5eb3bf8e44510bb4257b6" exitCode=0
Aug 03 23:36:05.001381 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:05.001383    2553 generic.go:356] "Generic (PLEG): container finished" podID="70c26563-c974-4779-b237-4412c5231119" containerID="7614c4e711a9b1c332e260a5d6c096623e9b9fc52fa605d8a0439d1d08ba2481" exitCode=0
Aug 03 23:36:05.001557 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:05.001389    2553 generic.go:356] "Generic (PLEG): container finished" podID="70c26563-c974-4779-b237-4412c5231119" containerID="0045283d9a71416e348411496b393ea8648710647edbef000622ee9aad3dd400" exitCode=0
Aug 03 23:36:05.001557 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:05.001412    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerDied","Data":"06ec45803a47e2af067c83d707cb7b256ff4cfa3876cc80de297659dc01ac1f4"}
Aug 03 23:36:05.001557 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:05.001473    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerDied","Data":"1ce3c9dc4970094bf6d2fae8f30a7c0e23cd9d2cd6c5eb3bf8e44510bb4257b6"}
Aug 03 23:36:05.001557 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:05.001501    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerDied","Data":"7614c4e711a9b1c332e260a5d6c096623e9b9fc52fa605d8a0439d1d08ba2481"}
Aug 03 23:36:05.001557 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:05.001525    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerDied","Data":"0045283d9a71416e348411496b393ea8648710647edbef000622ee9aad3dd400"}
Aug 03 23:36:06.008998 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.008971    2553 generic.go:356] "Generic (PLEG): container finished" podID="70c26563-c974-4779-b237-4412c5231119" containerID="8621c8ec797fc7466a0b0d4c5c75d7b0a1e58eb1715d1e81718bce15a255709a" exitCode=0
Aug 03 23:36:06.008998 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.008993    2553 generic.go:356] "Generic (PLEG): container finished" podID="70c26563-c974-4779-b237-4412c5231119" containerID="d943040d15613371c2e2406503f192b5a7a3e2275b3c08bb88bc558d6391a181" exitCode=0
Aug 03 23:36:06.009365 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.009012    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerDied","Data":"8621c8ec797fc7466a0b0d4c5c75d7b0a1e58eb1715d1e81718bce15a255709a"}
Aug 03 23:36:06.009365 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.009083    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerDied","Data":"d943040d15613371c2e2406503f192b5a7a3e2275b3c08bb88bc558d6391a181"}
Aug 03 23:36:06.102388 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.102368    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:06.189824 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.189796    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-web\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.189954 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.189834    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-main-tls\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.189954 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.189852    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-cluster-tls-config\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.189954 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.189882    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-alertmanager-trusted-ca-bundle\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.189954 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.189905    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-alertmanager-main-db\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.189954 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.189931    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-config-volume\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.189954 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.189948    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-metric\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.190266 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.189976    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-tls-assets\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.190266 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.190034    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvr8z\" (UniqueName: \"kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-kube-api-access-wvr8z\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.190266 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.190058    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-metrics-client-ca\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.190266 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.190109    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-web-config\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.190266 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.190157    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-config-out\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.190266 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.190182    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy\") pod \"70c26563-c974-4779-b237-4412c5231119\" (UID: \"70c26563-c974-4779-b237-4412c5231119\") "
Aug 03 23:36:06.190525 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.190257    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:36:06.190525 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.190465    2553 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.190685 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.190648    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 23:36:06.191435 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.191397    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:36:06.192910 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.192870    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:06.193193 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.193110    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-config-volume" (OuterVolumeSpecName: "config-volume") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:06.193582 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.193551    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:06.193803 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.193774    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:36:06.193803 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.193785    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:06.193803 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.193775    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:06.194011 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.193874    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-kube-api-access-wvr8z" (OuterVolumeSpecName: "kube-api-access-wvr8z") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "kube-api-access-wvr8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:36:06.194599 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.194578    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-config-out" (OuterVolumeSpecName: "config-out") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 23:36:06.197698 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.197676    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:06.205391 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.205370    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-web-config" (OuterVolumeSpecName: "web-config") pod "70c26563-c974-4779-b237-4412c5231119" (UID: "70c26563-c974-4779-b237-4412c5231119"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:06.290850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290827    2553 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-web-config\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.290850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290846    2553 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-config-out\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.290961 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290856    2553 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.290961 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290865    2553 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.290961 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290875    2553 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-main-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.290961 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290883    2553 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-cluster-tls-config\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.290961 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290891    2553 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/70c26563-c974-4779-b237-4412c5231119-alertmanager-main-db\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.290961 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290899    2553 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.290961 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290907    2553 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/70c26563-c974-4779-b237-4412c5231119-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.290961 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290916    2553 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-tls-assets\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.290961 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290924    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvr8z\" (UniqueName: \"kubernetes.io/projected/70c26563-c974-4779-b237-4412c5231119-kube-api-access-wvr8z\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.290961 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.290932    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70c26563-c974-4779-b237-4412c5231119-metrics-client-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:06.593613 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.593553    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:36:06.595669 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.595645    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e420d03-b0de-4f89-936a-cdafff2847a9-metrics-certs\") pod \"network-metrics-daemon-kld8m\" (UID: \"5e420d03-b0de-4f89-936a-cdafff2847a9\") " pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:36:06.663570 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.663549    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w6szp\""
Aug 03 23:36:06.671767 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.671752    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kld8m"
Aug 03 23:36:06.793516 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:06.793497    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kld8m"]
Aug 03 23:36:06.795892 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:36:06.795869    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e420d03_b0de_4f89_936a_cdafff2847a9.slice/crio-da6b5b5cd811720f0a579cc9a44aa8b59038251af17e6c49c1db5fe926c62c49 WatchSource:0}: Error finding container da6b5b5cd811720f0a579cc9a44aa8b59038251af17e6c49c1db5fe926c62c49: Status 404 returned error can't find the container with id da6b5b5cd811720f0a579cc9a44aa8b59038251af17e6c49c1db5fe926c62c49
Aug 03 23:36:07.014084 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.014049    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kld8m" event={"ID":"5e420d03-b0de-4f89-936a-cdafff2847a9","Type":"ContainerStarted","Data":"da6b5b5cd811720f0a579cc9a44aa8b59038251af17e6c49c1db5fe926c62c49"}
Aug 03 23:36:07.017104 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.017080    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c26563-c974-4779-b237-4412c5231119","Type":"ContainerDied","Data":"cb8f9545018acc59a6473093b46ee26d3315fe15b0851f1083c5d3d0d85f43e0"}
Aug 03 23:36:07.017104 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.017094    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.017230 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.017149    2553 scope.go:117] "RemoveContainer" containerID="06ec45803a47e2af067c83d707cb7b256ff4cfa3876cc80de297659dc01ac1f4"
Aug 03 23:36:07.024330 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.024315    2553 scope.go:117] "RemoveContainer" containerID="8621c8ec797fc7466a0b0d4c5c75d7b0a1e58eb1715d1e81718bce15a255709a"
Aug 03 23:36:07.030880 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.030860    2553 scope.go:117] "RemoveContainer" containerID="1ce3c9dc4970094bf6d2fae8f30a7c0e23cd9d2cd6c5eb3bf8e44510bb4257b6"
Aug 03 23:36:07.043730 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.038245    2553 scope.go:117] "RemoveContainer" containerID="d943040d15613371c2e2406503f192b5a7a3e2275b3c08bb88bc558d6391a181"
Aug 03 23:36:07.043730 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.039230    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 03 23:36:07.043889 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.043868    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 03 23:36:07.050914 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.050888    2553 scope.go:117] "RemoveContainer" containerID="7614c4e711a9b1c332e260a5d6c096623e9b9fc52fa605d8a0439d1d08ba2481"
Aug 03 23:36:07.058270 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.058252    2553 scope.go:117] "RemoveContainer" containerID="0045283d9a71416e348411496b393ea8648710647edbef000622ee9aad3dd400"
Aug 03 23:36:07.065259 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.065242    2553 scope.go:117] "RemoveContainer" containerID="1b77e7a2757d6edb7dabb56dfe9c5a232798410e96d77d96ca3628f62ea8f451"
Aug 03 23:36:07.065765 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.065746    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 03 23:36:07.066103 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066060    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39dc6b32-9e96-416c-90b6-7cee82af9363" containerName="console"
Aug 03 23:36:07.066103 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066076    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="39dc6b32-9e96-416c-90b6-7cee82af9363" containerName="console"
Aug 03 23:36:07.066103 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066088    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="config-reloader"
Aug 03 23:36:07.066103 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066096    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="config-reloader"
Aug 03 23:36:07.066228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066111    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy"
Aug 03 23:36:07.066228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066120    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy"
Aug 03 23:36:07.066228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066139    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy-metric"
Aug 03 23:36:07.066228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066147    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy-metric"
Aug 03 23:36:07.066228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066158    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="prom-label-proxy"
Aug 03 23:36:07.066228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066167    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="prom-label-proxy"
Aug 03 23:36:07.066228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066189    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy-web"
Aug 03 23:36:07.066228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066199    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy-web"
Aug 03 23:36:07.066228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066208    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="alertmanager"
Aug 03 23:36:07.066228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066216    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="alertmanager"
Aug 03 23:36:07.066228 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066226    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="init-config-reloader"
Aug 03 23:36:07.066539 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066234    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="init-config-reloader"
Aug 03 23:36:07.066539 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066286    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="39dc6b32-9e96-416c-90b6-7cee82af9363" containerName="console"
Aug 03 23:36:07.066539 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066292    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="config-reloader"
Aug 03 23:36:07.066539 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066298    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy-web"
Aug 03 23:36:07.066539 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066305    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy"
Aug 03 23:36:07.066539 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066312    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="prom-label-proxy"
Aug 03 23:36:07.066539 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066323    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="alertmanager"
Aug 03 23:36:07.066539 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.066328    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c26563-c974-4779-b237-4412c5231119" containerName="kube-rbac-proxy-metric"
Aug 03 23:36:07.069867 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.069839    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.072201 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.072182    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Aug 03 23:36:07.072285 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.072240    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cw6bw\""
Aug 03 23:36:07.072448 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.072426    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Aug 03 23:36:07.072559 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.072488    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Aug 03 23:36:07.072559 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.072506    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Aug 03 23:36:07.072559 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.072515    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Aug 03 23:36:07.072801 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.072616    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Aug 03 23:36:07.072801 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.072778    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Aug 03 23:36:07.072913 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.072796    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Aug 03 23:36:07.077665 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.077646    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Aug 03 23:36:07.083036 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.083019    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 03 23:36:07.097720 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.097683    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.097819 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.097736    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgvkt\" (UniqueName: \"kubernetes.io/projected/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-kube-api-access-lgvkt\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.097819 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.097777    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.097819 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.097799    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.097975 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.097821    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.097975 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.097845    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.097975 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.097865    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-config-out\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.097975 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.097889    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.097975 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.097931    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.097975 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.097965    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-web-config\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.098271 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.097988    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.098271 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.098028    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.098271 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.098051    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-config-volume\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.199294 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199228    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-web-config\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.199294 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199267    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.199294 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199293    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.199471 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199308    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-config-volume\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.199567 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199544    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.199691 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199573    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgvkt\" (UniqueName: \"kubernetes.io/projected/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-kube-api-access-lgvkt\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.199691 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199601    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.199691 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199633    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.199691 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199658    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.200052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199690    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.200052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199745    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-config-out\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.200052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199777    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.200052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.199807    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.200519 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.200491    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.200646 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.200623    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.201370 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.201346    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.202850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.202440    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-config-volume\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.202850 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.202837    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.202978 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.202934    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.202978 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.202955    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-config-out\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.203282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.203256    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.203478 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.203458    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.203560 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.203543    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-web-config\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.204099 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.204078    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.204337 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.204318    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.208502 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.208483    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgvkt\" (UniqueName: \"kubernetes.io/projected/7e492fc4-69ce-4cce-b5cd-74227c9cbdea-kube-api-access-lgvkt\") pod \"alertmanager-main-0\" (UID: \"7e492fc4-69ce-4cce-b5cd-74227c9cbdea\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.379877 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.379844    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 03 23:36:07.527320 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:07.527241    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 03 23:36:07.651816 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:36:07.651779    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e492fc4_69ce_4cce_b5cd_74227c9cbdea.slice/crio-7ebb676361bb4b8223c447a6e5cf19d797aba65cae109a6bc43ce70327c009d4 WatchSource:0}: Error finding container 7ebb676361bb4b8223c447a6e5cf19d797aba65cae109a6bc43ce70327c009d4: Status 404 returned error can't find the container with id 7ebb676361bb4b8223c447a6e5cf19d797aba65cae109a6bc43ce70327c009d4
Aug 03 23:36:08.024985 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.024902    2553 generic.go:356] "Generic (PLEG): container finished" podID="7e492fc4-69ce-4cce-b5cd-74227c9cbdea" containerID="86fdb9671862ccbed8a2707c26f8fdd3c82d776bbdb37cdd2dfece21bb79bb92" exitCode=0
Aug 03 23:36:08.025406 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.024982    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e492fc4-69ce-4cce-b5cd-74227c9cbdea","Type":"ContainerDied","Data":"86fdb9671862ccbed8a2707c26f8fdd3c82d776bbdb37cdd2dfece21bb79bb92"}
Aug 03 23:36:08.025406 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.025081    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e492fc4-69ce-4cce-b5cd-74227c9cbdea","Type":"ContainerStarted","Data":"7ebb676361bb4b8223c447a6e5cf19d797aba65cae109a6bc43ce70327c009d4"}
Aug 03 23:36:08.026755 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.026734    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kld8m" event={"ID":"5e420d03-b0de-4f89-936a-cdafff2847a9","Type":"ContainerStarted","Data":"43436e9991cab4239f44a1e427a7db3c04692d80c036995dc001e019075fe338"}
Aug 03 23:36:08.026883 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.026773    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kld8m" event={"ID":"5e420d03-b0de-4f89-936a-cdafff2847a9","Type":"ContainerStarted","Data":"33cb394f5334d4f4abb5feeff077cdc04fe546f1080ae96a9a44cfa0130c20c9"}
Aug 03 23:36:08.062026 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.061967    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kld8m" podStartSLOduration=375.185378822 podStartE2EDuration="6m16.061946514s" totalImagesPullingTime="876.567692ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:36:08.061605402 +0000 UTC m=+375.763312067" watchObservedRunningTime="2026-08-03 23:36:08.061946514 +0000 UTC m=+375.763653159"
Aug 03 23:36:08.763202 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.763172    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c26563-c974-4779-b237-4412c5231119" path="/var/lib/kubelet/pods/70c26563-c974-4779-b237-4412c5231119/volumes"
Aug 03 23:36:08.877529 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.877452    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"]
Aug 03 23:36:08.880580 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.880557    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:08.883081 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.883057    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Aug 03 23:36:08.883221 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.883066    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Aug 03 23:36:08.883221 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.883171    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Aug 03 23:36:08.883221 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.883212    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Aug 03 23:36:08.883498 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.883482    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Aug 03 23:36:08.884058 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.884041    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-n87kq\""
Aug 03 23:36:08.892213 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.892180    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Aug 03 23:36:08.893002 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.892973    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"]
Aug 03 23:36:08.914144 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.914105    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c6211f2-278b-4192-a4e9-1ca7f0a30797-metrics-client-ca\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:08.914282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.914149    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrzr2\" (UniqueName: \"kubernetes.io/projected/8c6211f2-278b-4192-a4e9-1ca7f0a30797-kube-api-access-nrzr2\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:08.914282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.914239    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-telemeter-client-tls\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:08.914282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.914270    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c6211f2-278b-4192-a4e9-1ca7f0a30797-serving-certs-ca-bundle\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:08.914454 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.914294    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-secret-telemeter-client\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:08.914454 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.914388    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:08.914454 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.914441    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-federate-client-tls\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:08.914581 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:08.914478    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c6211f2-278b-4192-a4e9-1ca7f0a30797-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.015608 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.015516    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-telemeter-client-tls\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.015608 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.015558    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c6211f2-278b-4192-a4e9-1ca7f0a30797-serving-certs-ca-bundle\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.015608 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.015590    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-secret-telemeter-client\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.015926 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.015628    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.015926 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.015669    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-federate-client-tls\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.015926 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.015720    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c6211f2-278b-4192-a4e9-1ca7f0a30797-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.015926 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.015771    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c6211f2-278b-4192-a4e9-1ca7f0a30797-metrics-client-ca\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.015926 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.015799    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrzr2\" (UniqueName: \"kubernetes.io/projected/8c6211f2-278b-4192-a4e9-1ca7f0a30797-kube-api-access-nrzr2\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.016405 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.016374    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c6211f2-278b-4192-a4e9-1ca7f0a30797-serving-certs-ca-bundle\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.016906 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.016882    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c6211f2-278b-4192-a4e9-1ca7f0a30797-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.017276 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.017251    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c6211f2-278b-4192-a4e9-1ca7f0a30797-metrics-client-ca\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.018543 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.018522    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-federate-client-tls\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.018543 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.018534    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-telemeter-client-tls\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.019129 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.019107    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-secret-telemeter-client\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.019199 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.019159    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c6211f2-278b-4192-a4e9-1ca7f0a30797-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.024460 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.024433    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrzr2\" (UniqueName: \"kubernetes.io/projected/8c6211f2-278b-4192-a4e9-1ca7f0a30797-kube-api-access-nrzr2\") pod \"telemeter-client-9667fbd4d-f4xx4\" (UID: \"8c6211f2-278b-4192-a4e9-1ca7f0a30797\") " pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.038490 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.038462    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e492fc4-69ce-4cce-b5cd-74227c9cbdea","Type":"ContainerStarted","Data":"53ef0f93db4a113677d99df07e9935abe60454614e1f6e91f9498357849d283c"}
Aug 03 23:36:09.038853 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.038540    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e492fc4-69ce-4cce-b5cd-74227c9cbdea","Type":"ContainerStarted","Data":"d92b03ddc15f92ca96e5c0d8765ae0d4af728b0e69ee0d859c069fed5cf1e6bc"}
Aug 03 23:36:09.038853 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.038594    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e492fc4-69ce-4cce-b5cd-74227c9cbdea","Type":"ContainerStarted","Data":"8061e70a50e06bdf77758cd89e2d91e31017c5c0845d76cb07561765703d2791"}
Aug 03 23:36:09.038853 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.038635    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e492fc4-69ce-4cce-b5cd-74227c9cbdea","Type":"ContainerStarted","Data":"6a3a17cb124e80bdd59e5c9da82da92effff945ebe21051457ad9df293b00c0f"}
Aug 03 23:36:09.038853 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.038674    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e492fc4-69ce-4cce-b5cd-74227c9cbdea","Type":"ContainerStarted","Data":"5399bf58099ad3efd18f16f65596bef8af088bf7a7cc12ca97cc7e045742df24"}
Aug 03 23:36:09.038853 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.038739    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e492fc4-69ce-4cce-b5cd-74227c9cbdea","Type":"ContainerStarted","Data":"7581a9fccdfe08bddccb02b8aea881976fa6645e9bb99a2ab2c5c864950a7cfd"}
Aug 03 23:36:09.062155 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.062110    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.044888398 podStartE2EDuration="2.062095783s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.207385ms" isStatefulPod=false podCreationTimestamp="2026-08-03 23:36:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:36:09.060104481 +0000 UTC m=+376.761811147" watchObservedRunningTime="2026-08-03 23:36:09.062095783 +0000 UTC m=+376.763802465"
Aug 03 23:36:09.174530 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.174496    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 23:36:09.174966 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.174783    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="prometheus" containerID="cri-o://16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863" gracePeriod=600
Aug 03 23:36:09.174966 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.174794    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy" containerID="cri-o://79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b" gracePeriod=600
Aug 03 23:36:09.174966 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.174832    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="thanos-sidecar" containerID="cri-o://9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32" gracePeriod=600
Aug 03 23:36:09.174966 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.174858    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="config-reloader" containerID="cri-o://2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d" gracePeriod=600
Aug 03 23:36:09.174966 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.174869    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy-web" containerID="cri-o://a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032" gracePeriod=600
Aug 03 23:36:09.174966 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.174942    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6" gracePeriod=600
Aug 03 23:36:09.200046 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.199851    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"
Aug 03 23:36:09.348238 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:09.348207    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9667fbd4d-f4xx4"]
Aug 03 23:36:09.351133 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:36:09.351106    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6211f2_278b_4192_a4e9_1ca7f0a30797.slice/crio-07376f9000c0c1b181ea2b743a943df3b2d8812ee6cb1c366f58e712c790d26c WatchSource:0}: Error finding container 07376f9000c0c1b181ea2b743a943df3b2d8812ee6cb1c366f58e712c790d26c: Status 404 returned error can't find the container with id 07376f9000c0c1b181ea2b743a943df3b2d8812ee6cb1c366f58e712c790d26c
Aug 03 23:36:10.044265 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.044230    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4" event={"ID":"8c6211f2-278b-4192-a4e9-1ca7f0a30797","Type":"ContainerStarted","Data":"07376f9000c0c1b181ea2b743a943df3b2d8812ee6cb1c366f58e712c790d26c"}
Aug 03 23:36:10.048431 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.048402    2553 generic.go:356] "Generic (PLEG): container finished" podID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerID="b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6" exitCode=0
Aug 03 23:36:10.048431 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.048430    2553 generic.go:356] "Generic (PLEG): container finished" podID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerID="79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b" exitCode=0
Aug 03 23:36:10.048621 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.048440    2553 generic.go:356] "Generic (PLEG): container finished" podID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerID="9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32" exitCode=0
Aug 03 23:36:10.048621 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.048446    2553 generic.go:356] "Generic (PLEG): container finished" podID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerID="2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d" exitCode=0
Aug 03 23:36:10.048621 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.048451    2553 generic.go:356] "Generic (PLEG): container finished" podID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerID="16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863" exitCode=0
Aug 03 23:36:10.048621 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.048439    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerDied","Data":"b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6"}
Aug 03 23:36:10.048621 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.048519    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerDied","Data":"79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b"}
Aug 03 23:36:10.048621 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.048551    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerDied","Data":"9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32"}
Aug 03 23:36:10.048621 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.048577    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerDied","Data":"2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d"}
Aug 03 23:36:10.048621 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.048602    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerDied","Data":"16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863"}
Aug 03 23:36:10.427247 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.427223    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:10.528979 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.528898    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config-out\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.528979 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.528936    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-metrics-client-certs\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.528979 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.528961    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-metrics-client-ca\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529232 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.528990    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-thanos-prometheus-http-client-file\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529232 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529019    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-db\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529232 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529060    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-serving-certs-ca-bundle\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529232 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529092    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-grpc-tls\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529232 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529113    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-0\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529232 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529136    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfqsm\" (UniqueName: \"kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-kube-api-access-sfqsm\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529232 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529192    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-tls\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529232 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529223    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-1\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529250    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-kubelet-serving-ca-bundle\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529282    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529305    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-web-config\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529354    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-2\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529392    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-kube-rbac-proxy\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529388    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:36:10.529601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529418    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:36:10.529601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529428    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529483    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529507    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-tls-assets\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.529601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529541    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-trusted-ca-bundle\") pod \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\" (UID: \"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8\") "
Aug 03 23:36:10.530197 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529639    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:36:10.530197 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529822    2553 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-metrics-client-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.530197 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529841    2553 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.530197 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.529858    2553 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.530197 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.530015    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:36:10.530695 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.530420    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:36:10.530695 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.530622    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 23:36:10.531359 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.531037    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:36:10.531845 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.531818    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:36:10.532072 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.532044    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:10.532138 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.532085    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:10.533217 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.533190    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config-out" (OuterVolumeSpecName: "config-out") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 23:36:10.533636 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.533612    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config" (OuterVolumeSpecName: "config") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:10.533946 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.533921    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:10.534254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.534219    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:10.534343 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.534253    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:36:10.534343 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.534290    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-kube-api-access-sfqsm" (OuterVolumeSpecName: "kube-api-access-sfqsm") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "kube-api-access-sfqsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:36:10.534562 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.534541    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:10.534618 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.534598    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:10.535039 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.535016    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:10.545003 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.544984    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-web-config" (OuterVolumeSpecName: "web-config") pod "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" (UID: "f5c1c722-24ba-4493-8fc6-80b58b5ca7b8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:36:10.630777 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630742    2553 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630777 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630776    2553 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-tls-assets\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630777 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630785    2553 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630793    2553 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config-out\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630802    2553 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-metrics-client-certs\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630810    2553 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630819    2553 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-db\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630827    2553 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-grpc-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630837    2553 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630844    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfqsm\" (UniqueName: \"kubernetes.io/projected/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-kube-api-access-sfqsm\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630852    2553 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630860    2553 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630869    2553 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-config\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630876    2553 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-web-config\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630884    2553 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630891    2553 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-kube-rbac-proxy\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:10.630945 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:10.630899    2553 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:36:11.058465 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.058422    2553 generic.go:356] "Generic (PLEG): container finished" podID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerID="a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032" exitCode=0
Aug 03 23:36:11.058465 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.058469    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerDied","Data":"a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032"}
Aug 03 23:36:11.058955 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.058505    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.058955 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.058531    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5c1c722-24ba-4493-8fc6-80b58b5ca7b8","Type":"ContainerDied","Data":"7f7f82eba45761075d834ff6b3ad71f96436fe160be6d2509b773bd3d7235df0"}
Aug 03 23:36:11.058955 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.058565    2553 scope.go:117] "RemoveContainer" containerID="b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6"
Aug 03 23:36:11.084950 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.084918    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 23:36:11.088846 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.088826    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 23:36:11.112147 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112126    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 23:36:11.112458 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112444    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="thanos-sidecar"
Aug 03 23:36:11.112531 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112462    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="thanos-sidecar"
Aug 03 23:36:11.112531 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112475    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy-thanos"
Aug 03 23:36:11.112531 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112482    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy-thanos"
Aug 03 23:36:11.112531 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112504    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="init-config-reloader"
Aug 03 23:36:11.112531 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112512    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="init-config-reloader"
Aug 03 23:36:11.112531 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112522    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="prometheus"
Aug 03 23:36:11.112531 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112530    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="prometheus"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112545    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy-web"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112554    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy-web"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112565    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="config-reloader"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112574    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="config-reloader"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112592    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112601    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112675    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112687    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="thanos-sidecar"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112697    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy-web"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112729    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="config-reloader"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112738    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="prometheus"
Aug 03 23:36:11.112939 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.112748    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" containerName="kube-rbac-proxy-thanos"
Aug 03 23:36:11.115594 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.115577    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.118506 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.118484    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Aug 03 23:36:11.118636 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.118483    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Aug 03 23:36:11.118697 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.118675    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-297s8milb60dv\""
Aug 03 23:36:11.118878 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.118844    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Aug 03 23:36:11.119088 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.119073    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Aug 03 23:36:11.119193 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.119136    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Aug 03 23:36:11.119193 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.119143    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Aug 03 23:36:11.119193 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.119169    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Aug 03 23:36:11.119193 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.119180    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Aug 03 23:36:11.119390 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.119289    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mdpdv\""
Aug 03 23:36:11.119390 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.119324    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Aug 03 23:36:11.119390 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.119365    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Aug 03 23:36:11.119390 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.119380    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Aug 03 23:36:11.119954 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.119933    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Aug 03 23:36:11.120054 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.119904    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Aug 03 23:36:11.122270 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.122239    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Aug 03 23:36:11.124076 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.124050    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Aug 03 23:36:11.130591 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.130571    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 23:36:11.235647 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235572    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.235647 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235608    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.235884 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235664    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.235884 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235690    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.235884 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235738    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.235884 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235778    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-web-config\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.235884 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235827    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.235884 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235858    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236135 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235887    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7pkt\" (UniqueName: \"kubernetes.io/projected/73ea3c88-2594-443b-a799-1ef0c82103c1-kube-api-access-l7pkt\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236135 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235915    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236135 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235939    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236135 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.235967    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236135 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.236033    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236135 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.236064    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73ea3c88-2594-443b-a799-1ef0c82103c1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236135 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.236091    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236135 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.236120    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236439 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.236143    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236439 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.236166    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236439 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.236192    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73ea3c88-2594-443b-a799-1ef0c82103c1-config-out\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.236439 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.236220    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-config\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.336993 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.336950    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337156 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337009    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337213 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337189    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337258 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337245    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-web-config\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337326 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337300    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337375 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337329    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337375 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337352    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7pkt\" (UniqueName: \"kubernetes.io/projected/73ea3c88-2594-443b-a799-1ef0c82103c1-kube-api-access-l7pkt\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337469 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337379    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337469 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337408    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337469 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337460    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337617 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337495    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337617 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337525    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73ea3c88-2594-443b-a799-1ef0c82103c1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337617 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337550    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337617 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337590    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337841 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337627    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337841 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337651    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337841 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337680    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73ea3c88-2594-443b-a799-1ef0c82103c1-config-out\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.337841 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337806    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-config\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.338022 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337842    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.338022 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.337868    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.338428 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.338389    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.338543 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.338444    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.338673 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.338651    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.339330 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.339303    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.340404 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.340373    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.341254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.340950    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.341254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.341069    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.341254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.341115    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-web-config\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.341254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.341178    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.341254 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.341211    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.341577 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.341299    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.341577 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.341404    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.341679 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.341657    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/73ea3c88-2594-443b-a799-1ef0c82103c1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.342493 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.342466    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.343450 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.343409    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73ea3c88-2594-443b-a799-1ef0c82103c1-config-out\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.343972 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.343951    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-config\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.344292 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.344244    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.344292 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.344274    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73ea3c88-2594-443b-a799-1ef0c82103c1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.344402 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.344306    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/73ea3c88-2594-443b-a799-1ef0c82103c1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.347283 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.347261    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7pkt\" (UniqueName: \"kubernetes.io/projected/73ea3c88-2594-443b-a799-1ef0c82103c1-kube-api-access-l7pkt\") pod \"prometheus-k8s-0\" (UID: \"73ea3c88-2594-443b-a799-1ef0c82103c1\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.428202 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.428168    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:36:11.464227 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.464208    2553 scope.go:117] "RemoveContainer" containerID="79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b"
Aug 03 23:36:11.487239 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.487213    2553 scope.go:117] "RemoveContainer" containerID="a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032"
Aug 03 23:36:11.494648 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.494620    2553 scope.go:117] "RemoveContainer" containerID="9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32"
Aug 03 23:36:11.503589 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.503415    2553 scope.go:117] "RemoveContainer" containerID="2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d"
Aug 03 23:36:11.514331 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.514246    2553 scope.go:117] "RemoveContainer" containerID="16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863"
Aug 03 23:36:11.524526 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.524506    2553 scope.go:117] "RemoveContainer" containerID="ca4f0cf080dd565459af099d663cd94679b573b519ca171023e0673cdb80e71e"
Aug 03 23:36:11.541181 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.541149    2553 scope.go:117] "RemoveContainer" containerID="b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6"
Aug 03 23:36:11.541439 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:36:11.541404    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6\": container with ID starting with b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6 not found: ID does not exist" containerID="b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6"
Aug 03 23:36:11.541539 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.541437    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6"} err="failed to get container status \"b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6\": rpc error: code = NotFound desc = could not find container \"b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6\": container with ID starting with b7ac8fb3077bd018759f3124e138a9da5f9472e7721847410222d5d7e3a4cba6 not found: ID does not exist"
Aug 03 23:36:11.541539 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.541462    2553 scope.go:117] "RemoveContainer" containerID="79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b"
Aug 03 23:36:11.541794 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:36:11.541776    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b\": container with ID starting with 79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b not found: ID does not exist" containerID="79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b"
Aug 03 23:36:11.541866 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.541801    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b"} err="failed to get container status \"79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b\": rpc error: code = NotFound desc = could not find container \"79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b\": container with ID starting with 79afff291e3dfb1e24acd2a77f4fa6d4cf558899eec7601f9892d4ef312a874b not found: ID does not exist"
Aug 03 23:36:11.541866 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.541819    2553 scope.go:117] "RemoveContainer" containerID="a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032"
Aug 03 23:36:11.542065 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:36:11.542049    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032\": container with ID starting with a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032 not found: ID does not exist" containerID="a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032"
Aug 03 23:36:11.542120 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.542071    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032"} err="failed to get container status \"a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032\": rpc error: code = NotFound desc = could not find container \"a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032\": container with ID starting with a5c16607e54dcf955eaf4f71e0f19c77ffef6771e5b85d65133cbc2e3191a032 not found: ID does not exist"
Aug 03 23:36:11.542120 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.542086    2553 scope.go:117] "RemoveContainer" containerID="9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32"
Aug 03 23:36:11.542397 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:36:11.542362    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32\": container with ID starting with 9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32 not found: ID does not exist" containerID="9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32"
Aug 03 23:36:11.542397 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.542386    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32"} err="failed to get container status \"9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32\": rpc error: code = NotFound desc = could not find container \"9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32\": container with ID starting with 9d32b5f7563d6114eab21e7a6b5d32c503308ab2126337c002dc9b8c2b3adf32 not found: ID does not exist"
Aug 03 23:36:11.542595 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.542405    2553 scope.go:117] "RemoveContainer" containerID="2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d"
Aug 03 23:36:11.542756 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:36:11.542731    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d\": container with ID starting with 2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d not found: ID does not exist" containerID="2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d"
Aug 03 23:36:11.542818 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.542761    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d"} err="failed to get container status \"2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d\": rpc error: code = NotFound desc = could not find container \"2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d\": container with ID starting with 2199ee690259ceb718cf265de13e3909d7a1a5b2505ad8a3d6ed314ea8fd5f7d not found: ID does not exist"
Aug 03 23:36:11.542818 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.542777    2553 scope.go:117] "RemoveContainer" containerID="16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863"
Aug 03 23:36:11.543067 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:36:11.543046    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863\": container with ID starting with 16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863 not found: ID does not exist" containerID="16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863"
Aug 03 23:36:11.543179 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.543068    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863"} err="failed to get container status \"16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863\": rpc error: code = NotFound desc = could not find container \"16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863\": container with ID starting with 16645f3aa438c2b7a8572b00459ffe35dfc5abe108c4ec58993114a897d2e863 not found: ID does not exist"
Aug 03 23:36:11.543179 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.543085    2553 scope.go:117] "RemoveContainer" containerID="ca4f0cf080dd565459af099d663cd94679b573b519ca171023e0673cdb80e71e"
Aug 03 23:36:11.543359 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:36:11.543336    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca4f0cf080dd565459af099d663cd94679b573b519ca171023e0673cdb80e71e\": container with ID starting with ca4f0cf080dd565459af099d663cd94679b573b519ca171023e0673cdb80e71e not found: ID does not exist" containerID="ca4f0cf080dd565459af099d663cd94679b573b519ca171023e0673cdb80e71e"
Aug 03 23:36:11.543417 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.543364    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca4f0cf080dd565459af099d663cd94679b573b519ca171023e0673cdb80e71e"} err="failed to get container status \"ca4f0cf080dd565459af099d663cd94679b573b519ca171023e0673cdb80e71e\": rpc error: code = NotFound desc = could not find container \"ca4f0cf080dd565459af099d663cd94679b573b519ca171023e0673cdb80e71e\": container with ID starting with ca4f0cf080dd565459af099d663cd94679b573b519ca171023e0673cdb80e71e not found: ID does not exist"
Aug 03 23:36:11.642924 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:11.642894    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 23:36:11.644819 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:36:11.644793    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ea3c88_2594_443b_a799_1ef0c82103c1.slice/crio-96b83b87678c650e060afc778675a2c1cc648e2b0465672c55c079e385df0c7e WatchSource:0}: Error finding container 96b83b87678c650e060afc778675a2c1cc648e2b0465672c55c079e385df0c7e: Status 404 returned error can't find the container with id 96b83b87678c650e060afc778675a2c1cc648e2b0465672c55c079e385df0c7e
Aug 03 23:36:12.065220 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:12.065190    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4" event={"ID":"8c6211f2-278b-4192-a4e9-1ca7f0a30797","Type":"ContainerStarted","Data":"58685b98509d48bea19ae7b21a8ad4f48a0f42b1c679d4c5fa967489a5c1a489"}
Aug 03 23:36:12.065603 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:12.065246    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4" event={"ID":"8c6211f2-278b-4192-a4e9-1ca7f0a30797","Type":"ContainerStarted","Data":"44a5db7c00d7afb573d304345f4fc94fb8e8f4c71df5b66a04b5325457b49e49"}
Aug 03 23:36:12.065603 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:12.065268    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4" event={"ID":"8c6211f2-278b-4192-a4e9-1ca7f0a30797","Type":"ContainerStarted","Data":"db2054435c3199ab0ff6f3a4cb8e93dd0843ffd01ee4d646fef8cfdb5ee1ed40"}
Aug 03 23:36:12.066595 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:12.066573    2553 generic.go:356] "Generic (PLEG): container finished" podID="73ea3c88-2594-443b-a799-1ef0c82103c1" containerID="527d326d431b6c426002bc9a26f4837e19dbcf9a9223d6b110e7db2d4cf19560" exitCode=0
Aug 03 23:36:12.066700 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:12.066659    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73ea3c88-2594-443b-a799-1ef0c82103c1","Type":"ContainerDied","Data":"527d326d431b6c426002bc9a26f4837e19dbcf9a9223d6b110e7db2d4cf19560"}
Aug 03 23:36:12.066785 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:12.066748    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73ea3c88-2594-443b-a799-1ef0c82103c1","Type":"ContainerStarted","Data":"96b83b87678c650e060afc778675a2c1cc648e2b0465672c55c079e385df0c7e"}
Aug 03 23:36:12.088009 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:12.087944    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9667fbd4d-f4xx4" podStartSLOduration=1.9455913360000001 podStartE2EDuration="4.087923825s" totalImagesPullingTime="2.142332489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:36:12.085633082 +0000 UTC m=+379.787339748" watchObservedRunningTime="2026-08-03 23:36:12.087923825 +0000 UTC m=+379.789630491"
Aug 03 23:36:12.763441 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:12.763405    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5c1c722-24ba-4493-8fc6-80b58b5ca7b8" path="/var/lib/kubelet/pods/f5c1c722-24ba-4493-8fc6-80b58b5ca7b8/volumes"
Aug 03 23:36:13.076770 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:13.076646    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73ea3c88-2594-443b-a799-1ef0c82103c1","Type":"ContainerStarted","Data":"bf92eee0b06c56c5b8eb14ffbd37f29144797b4da8e6a1e4aa77fcf3e73ad5c5"}
Aug 03 23:36:13.076770 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:13.076753    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73ea3c88-2594-443b-a799-1ef0c82103c1","Type":"ContainerStarted","Data":"90f1763a462453b1d3026d4a5e866a6034ba5182cee2d173a240cf507d02d516"}
Aug 03 23:36:13.077171 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:13.076797    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73ea3c88-2594-443b-a799-1ef0c82103c1","Type":"ContainerStarted","Data":"e91bc6c71b3b3853888e8d2c82a91d26e9991f7ccffe496f94ebf0ebe15bfc63"}
Aug 03 23:36:13.077171 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:13.076839    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73ea3c88-2594-443b-a799-1ef0c82103c1","Type":"ContainerStarted","Data":"da1d3eb589d4fd476e4a70f32e2cd92debe11941a4aec7cae573bf0fec295719"}
Aug 03 23:36:13.077171 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:13.076874    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73ea3c88-2594-443b-a799-1ef0c82103c1","Type":"ContainerStarted","Data":"be22da9c0c1f4894b7f54311bd6b0b6acb360de9122c2dfd0b35c02a7e5bf330"}
Aug 03 23:36:13.077171 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:13.076912    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73ea3c88-2594-443b-a799-1ef0c82103c1","Type":"ContainerStarted","Data":"c5f92bb5964461843db131774ef95ed16f50c820d9a6562ec9654b1f9ef829ee"}
Aug 03 23:36:13.107940 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:13.107557    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.086891522 podStartE2EDuration="2.107536258s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.644736ms" isStatefulPod=false podCreationTimestamp="2026-08-03 23:36:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:36:13.106305225 +0000 UTC m=+380.808011890" watchObservedRunningTime="2026-08-03 23:36:13.107536258 +0000 UTC m=+380.809242923"
Aug 03 23:36:16.429000 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:36:16.428964    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:37:11.428598 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:37:11.428563    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:37:11.444065 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:37:11.444042    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:37:12.322862 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:37:12.322831    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 23:39:28.665979 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.665948    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"]
Aug 03 23:39:28.669181 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.669166    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.672352 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.672322    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 03 23:39:28.672464 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.672332    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 03 23:39:28.672464 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.672340    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 03 23:39:28.672566 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.672521    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 03 23:39:28.673165 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.672945    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hcprk\""
Aug 03 23:39:28.681885 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.681866    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"]
Aug 03 23:39:28.775553 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.775532    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/29498929-93a1-4db4-9b8e-d3d0c3deb906-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.775659 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.775558    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bt2z\" (UniqueName: \"kubernetes.io/projected/29498929-93a1-4db4-9b8e-d3d0c3deb906-kube-api-access-2bt2z\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.775659 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.775584    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/29498929-93a1-4db4-9b8e-d3d0c3deb906-webhook-cert\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.775659 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.775639    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/29498929-93a1-4db4-9b8e-d3d0c3deb906-manifests\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.775804 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.775676    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29498929-93a1-4db4-9b8e-d3d0c3deb906-tmp\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.876811 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.876789    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/29498929-93a1-4db4-9b8e-d3d0c3deb906-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.876901 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.876815    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bt2z\" (UniqueName: \"kubernetes.io/projected/29498929-93a1-4db4-9b8e-d3d0c3deb906-kube-api-access-2bt2z\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.876901 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.876842    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/29498929-93a1-4db4-9b8e-d3d0c3deb906-webhook-cert\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.876981 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.876964    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/29498929-93a1-4db4-9b8e-d3d0c3deb906-manifests\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.877026 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.877015    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29498929-93a1-4db4-9b8e-d3d0c3deb906-tmp\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.877303 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.877281    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/29498929-93a1-4db4-9b8e-d3d0c3deb906-manifests\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.879283 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.879258    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29498929-93a1-4db4-9b8e-d3d0c3deb906-tmp\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.879358 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.879308    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/29498929-93a1-4db4-9b8e-d3d0c3deb906-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.879358 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.879336    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/29498929-93a1-4db4-9b8e-d3d0c3deb906-webhook-cert\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.884796 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.884777    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bt2z\" (UniqueName: \"kubernetes.io/projected/29498929-93a1-4db4-9b8e-d3d0c3deb906-kube-api-access-2bt2z\") pod \"opendatahub-operator-controller-manager-6869c6d9bb-b2lh2\" (UID: \"29498929-93a1-4db4-9b8e-d3d0c3deb906\") " pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:28.981868 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:28.981850    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:29.109516 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:29.109492    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"]
Aug 03 23:39:29.112237 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:39:29.112211    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29498929_93a1_4db4_9b8e_d3d0c3deb906.slice/crio-7ab6b1f6edcbe86eed692d42a6b7ca1359862ab1664e9c594d4b40cd5cab899c WatchSource:0}: Error finding container 7ab6b1f6edcbe86eed692d42a6b7ca1359862ab1664e9c594d4b40cd5cab899c: Status 404 returned error can't find the container with id 7ab6b1f6edcbe86eed692d42a6b7ca1359862ab1664e9c594d4b40cd5cab899c
Aug 03 23:39:29.835574 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:29.835536    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2" event={"ID":"29498929-93a1-4db4-9b8e-d3d0c3deb906","Type":"ContainerStarted","Data":"7ab6b1f6edcbe86eed692d42a6b7ca1359862ab1664e9c594d4b40cd5cab899c"}
Aug 03 23:39:32.849952 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:32.849917    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2" event={"ID":"29498929-93a1-4db4-9b8e-d3d0c3deb906","Type":"ContainerStarted","Data":"0101986cf02cc9dd83e577442e34b7f3361f4448fc89cfd37d2a1d26b0c205e0"}
Aug 03 23:39:33.855435 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:33.855401    2553 generic.go:356] "Generic (PLEG): container finished" podID="29498929-93a1-4db4-9b8e-d3d0c3deb906" containerID="0101986cf02cc9dd83e577442e34b7f3361f4448fc89cfd37d2a1d26b0c205e0" exitCode=0
Aug 03 23:39:33.855913 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:33.855463    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2" event={"ID":"29498929-93a1-4db4-9b8e-d3d0c3deb906","Type":"ContainerDied","Data":"0101986cf02cc9dd83e577442e34b7f3361f4448fc89cfd37d2a1d26b0c205e0"}
Aug 03 23:39:34.861293 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:34.861256    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2" event={"ID":"29498929-93a1-4db4-9b8e-d3d0c3deb906","Type":"ContainerStarted","Data":"e449ed411c597dff772bd57480613e4b3d852e82444b1730101e31d485c81f0d"}
Aug 03 23:39:34.861664 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:34.861506    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:34.884482 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:34.884441    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2" podStartSLOduration=2.548881029 podStartE2EDuration="6.884428241s" totalImagesPullingTime="3.398113392s" totalInitContainerRuntime="937.43382ms" isStatefulPod=false podCreationTimestamp="2026-08-03 23:39:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:39:34.88367256 +0000 UTC m=+582.585379222" watchObservedRunningTime="2026-08-03 23:39:34.884428241 +0000 UTC m=+582.586134903"
Aug 03 23:39:45.868036 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:45.868009    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6869c6d9bb-b2lh2"
Aug 03 23:39:48.196153 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.196124    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"]
Aug 03 23:39:48.198354 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.198335    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.201981 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.201947    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Aug 03 23:39:48.201981 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.201971    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7rc2k\""
Aug 03 23:39:48.202158 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.201983    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Aug 03 23:39:48.202158 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.202025    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Aug 03 23:39:48.202158 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.202049    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Aug 03 23:39:48.202158 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.201986    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Aug 03 23:39:48.208591 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.208570    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"]
Aug 03 23:39:48.332568 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.332542    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d347fef-80d3-4772-b33f-5202aa41f86a-cert\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.332695 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.332587    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1d347fef-80d3-4772-b33f-5202aa41f86a-metrics-cert\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.332695 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.332608    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnrtw\" (UniqueName: \"kubernetes.io/projected/1d347fef-80d3-4772-b33f-5202aa41f86a-kube-api-access-qnrtw\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.332695 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.332687    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1d347fef-80d3-4772-b33f-5202aa41f86a-manager-config\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.433448 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.433422    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1d347fef-80d3-4772-b33f-5202aa41f86a-manager-config\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.433556 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.433466    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d347fef-80d3-4772-b33f-5202aa41f86a-cert\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.433556 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.433494    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1d347fef-80d3-4772-b33f-5202aa41f86a-metrics-cert\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.433556 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.433521    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnrtw\" (UniqueName: \"kubernetes.io/projected/1d347fef-80d3-4772-b33f-5202aa41f86a-kube-api-access-qnrtw\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.434094 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.434069    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1d347fef-80d3-4772-b33f-5202aa41f86a-manager-config\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.435946 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.435925    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1d347fef-80d3-4772-b33f-5202aa41f86a-metrics-cert\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.436035 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.435984    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d347fef-80d3-4772-b33f-5202aa41f86a-cert\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.444993 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.444960    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnrtw\" (UniqueName: \"kubernetes.io/projected/1d347fef-80d3-4772-b33f-5202aa41f86a-kube-api-access-qnrtw\") pod \"lws-controller-manager-d64bf5b4d-6j4gt\" (UID: \"1d347fef-80d3-4772-b33f-5202aa41f86a\") " pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.506809 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.506753    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:48.636218 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.636115    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"]
Aug 03 23:39:48.638267 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:39:48.638245    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d347fef_80d3_4772_b33f_5202aa41f86a.slice/crio-6909546ed7306dc3579cfc6bd6be8a492c32185bb0e966e5e0908d612ae95d95 WatchSource:0}: Error finding container 6909546ed7306dc3579cfc6bd6be8a492c32185bb0e966e5e0908d612ae95d95: Status 404 returned error can't find the container with id 6909546ed7306dc3579cfc6bd6be8a492c32185bb0e966e5e0908d612ae95d95
Aug 03 23:39:48.922814 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:48.922738    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt" event={"ID":"1d347fef-80d3-4772-b33f-5202aa41f86a","Type":"ContainerStarted","Data":"6909546ed7306dc3579cfc6bd6be8a492c32185bb0e966e5e0908d612ae95d95"}
Aug 03 23:39:51.939054 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:51.939015    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt" event={"ID":"1d347fef-80d3-4772-b33f-5202aa41f86a","Type":"ContainerStarted","Data":"ea4a4ccbf6015dddfc26f31f6e651df089d151afce97c34b3542145ce635fd48"}
Aug 03 23:39:51.939432 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:51.939182    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:39:51.956043 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:51.955999    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt" podStartSLOduration=1.651867394 podStartE2EDuration="3.955985525s" totalImagesPullingTime="2.304118131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:39:51.955568895 +0000 UTC m=+599.657275552" watchObservedRunningTime="2026-08-03 23:39:51.955985525 +0000 UTC m=+599.657692188"
Aug 03 23:39:53.681073 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.681033    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"]
Aug 03 23:39:53.683052 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.683037    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"
Aug 03 23:39:53.686455 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.686437    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 03 23:39:53.686582 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.686535    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dp2t8\""
Aug 03 23:39:53.686582 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.686556    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 03 23:39:53.692057 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.692037    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"]
Aug 03 23:39:53.777023 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.776979    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt42x\" (UniqueName: \"kubernetes.io/projected/4a7db60a-8023-4e79-a1c4-9066642292d4-kube-api-access-nt42x\") pod \"kube-auth-proxy-6bd58c6859-d5n7t\" (UID: \"4a7db60a-8023-4e79-a1c4-9066642292d4\") " pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"
Aug 03 23:39:53.777157 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.777029    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a7db60a-8023-4e79-a1c4-9066642292d4-tmp\") pod \"kube-auth-proxy-6bd58c6859-d5n7t\" (UID: \"4a7db60a-8023-4e79-a1c4-9066642292d4\") " pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"
Aug 03 23:39:53.777157 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.777087    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a7db60a-8023-4e79-a1c4-9066642292d4-tls-certs\") pod \"kube-auth-proxy-6bd58c6859-d5n7t\" (UID: \"4a7db60a-8023-4e79-a1c4-9066642292d4\") " pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"
Aug 03 23:39:53.877406 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.877382    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt42x\" (UniqueName: \"kubernetes.io/projected/4a7db60a-8023-4e79-a1c4-9066642292d4-kube-api-access-nt42x\") pod \"kube-auth-proxy-6bd58c6859-d5n7t\" (UID: \"4a7db60a-8023-4e79-a1c4-9066642292d4\") " pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"
Aug 03 23:39:53.877506 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.877411    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a7db60a-8023-4e79-a1c4-9066642292d4-tmp\") pod \"kube-auth-proxy-6bd58c6859-d5n7t\" (UID: \"4a7db60a-8023-4e79-a1c4-9066642292d4\") " pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"
Aug 03 23:39:53.877506 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.877452    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a7db60a-8023-4e79-a1c4-9066642292d4-tls-certs\") pod \"kube-auth-proxy-6bd58c6859-d5n7t\" (UID: \"4a7db60a-8023-4e79-a1c4-9066642292d4\") " pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"
Aug 03 23:39:53.879811 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.879789    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a7db60a-8023-4e79-a1c4-9066642292d4-tmp\") pod \"kube-auth-proxy-6bd58c6859-d5n7t\" (UID: \"4a7db60a-8023-4e79-a1c4-9066642292d4\") " pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"
Aug 03 23:39:53.879968 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.879949    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a7db60a-8023-4e79-a1c4-9066642292d4-tls-certs\") pod \"kube-auth-proxy-6bd58c6859-d5n7t\" (UID: \"4a7db60a-8023-4e79-a1c4-9066642292d4\") " pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"
Aug 03 23:39:53.884994 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.884966    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt42x\" (UniqueName: \"kubernetes.io/projected/4a7db60a-8023-4e79-a1c4-9066642292d4-kube-api-access-nt42x\") pod \"kube-auth-proxy-6bd58c6859-d5n7t\" (UID: \"4a7db60a-8023-4e79-a1c4-9066642292d4\") " pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"
Aug 03 23:39:53.993015 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:53.992994    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"
Aug 03 23:39:54.116780 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:54.116758    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t"]
Aug 03 23:39:54.119456 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:39:54.119430    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a7db60a_8023_4e79_a1c4_9066642292d4.slice/crio-9dff6362678c3d7a19e78aeaed215c4f709b6aed675012893b61e58e6917076d WatchSource:0}: Error finding container 9dff6362678c3d7a19e78aeaed215c4f709b6aed675012893b61e58e6917076d: Status 404 returned error can't find the container with id 9dff6362678c3d7a19e78aeaed215c4f709b6aed675012893b61e58e6917076d
Aug 03 23:39:54.121443 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:54.121429    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 23:39:54.953230 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:54.953195    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t" event={"ID":"4a7db60a-8023-4e79-a1c4-9066642292d4","Type":"ContainerStarted","Data":"9dff6362678c3d7a19e78aeaed215c4f709b6aed675012893b61e58e6917076d"}
Aug 03 23:39:56.963871 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:56.963839    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t" event={"ID":"4a7db60a-8023-4e79-a1c4-9066642292d4","Type":"ContainerStarted","Data":"96f8f21f61f9c3c0386a6d70055448ea519bc8b3c0b6be74770eb7c087f1bdec"}
Aug 03 23:39:56.988053 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:39:56.988008    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bd58c6859-d5n7t" podStartSLOduration=1.242104216 podStartE2EDuration="3.987993943s" totalImagesPullingTime="2.745889727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:39:56.986336151 +0000 UTC m=+604.688042812" watchObservedRunningTime="2026-08-03 23:39:56.987993943 +0000 UTC m=+604.689700605"
Aug 03 23:40:02.944839 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:40:02.944808    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d64bf5b4d-6j4gt"
Aug 03 23:41:13.335753 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.335701    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"]
Aug 03 23:41:13.338600 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.338584    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"
Aug 03 23:41:13.340953 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.340924    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Aug 03 23:41:13.341081 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.340950    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-m8rpf\""
Aug 03 23:41:13.341081 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.340950    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Aug 03 23:41:13.342039 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.342022    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 03 23:41:13.342129 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.342023    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 03 23:41:13.350881 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.350862    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"]
Aug 03 23:41:13.422844 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.422805    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpz92\" (UniqueName: \"kubernetes.io/projected/fec9d7c3-08e6-4473-a842-e3acee90dac6-kube-api-access-vpz92\") pod \"kuadrant-console-plugin-598f86468b-pwks2\" (UID: \"fec9d7c3-08e6-4473-a842-e3acee90dac6\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"
Aug 03 23:41:13.422844 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.422846    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fec9d7c3-08e6-4473-a842-e3acee90dac6-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-pwks2\" (UID: \"fec9d7c3-08e6-4473-a842-e3acee90dac6\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"
Aug 03 23:41:13.423071 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.422922    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fec9d7c3-08e6-4473-a842-e3acee90dac6-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-pwks2\" (UID: \"fec9d7c3-08e6-4473-a842-e3acee90dac6\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"
Aug 03 23:41:13.523916 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.523879    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fec9d7c3-08e6-4473-a842-e3acee90dac6-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-pwks2\" (UID: \"fec9d7c3-08e6-4473-a842-e3acee90dac6\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"
Aug 03 23:41:13.524085 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.523950    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fec9d7c3-08e6-4473-a842-e3acee90dac6-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-pwks2\" (UID: \"fec9d7c3-08e6-4473-a842-e3acee90dac6\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"
Aug 03 23:41:13.524085 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.524035    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpz92\" (UniqueName: \"kubernetes.io/projected/fec9d7c3-08e6-4473-a842-e3acee90dac6-kube-api-access-vpz92\") pod \"kuadrant-console-plugin-598f86468b-pwks2\" (UID: \"fec9d7c3-08e6-4473-a842-e3acee90dac6\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"
Aug 03 23:41:13.524656 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.524625    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fec9d7c3-08e6-4473-a842-e3acee90dac6-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-pwks2\" (UID: \"fec9d7c3-08e6-4473-a842-e3acee90dac6\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"
Aug 03 23:41:13.526437 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.526419    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fec9d7c3-08e6-4473-a842-e3acee90dac6-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-pwks2\" (UID: \"fec9d7c3-08e6-4473-a842-e3acee90dac6\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"
Aug 03 23:41:13.532214 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.532189    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpz92\" (UniqueName: \"kubernetes.io/projected/fec9d7c3-08e6-4473-a842-e3acee90dac6-kube-api-access-vpz92\") pod \"kuadrant-console-plugin-598f86468b-pwks2\" (UID: \"fec9d7c3-08e6-4473-a842-e3acee90dac6\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"
Aug 03 23:41:13.647328 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.647240    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"
Aug 03 23:41:13.777590 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:13.777567    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2"]
Aug 03 23:41:13.780102 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:41:13.780081    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfec9d7c3_08e6_4473_a842_e3acee90dac6.slice/crio-7971586408d8c7d9bf0f7a7b5bee1e9b51dc3bfbd6863230bb3988a4ef204d19 WatchSource:0}: Error finding container 7971586408d8c7d9bf0f7a7b5bee1e9b51dc3bfbd6863230bb3988a4ef204d19: Status 404 returned error can't find the container with id 7971586408d8c7d9bf0f7a7b5bee1e9b51dc3bfbd6863230bb3988a4ef204d19
Aug 03 23:41:14.291567 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:14.291535    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2" event={"ID":"fec9d7c3-08e6-4473-a842-e3acee90dac6","Type":"ContainerStarted","Data":"7971586408d8c7d9bf0f7a7b5bee1e9b51dc3bfbd6863230bb3988a4ef204d19"}
Aug 03 23:41:19.315676 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:19.315638    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2" event={"ID":"fec9d7c3-08e6-4473-a842-e3acee90dac6","Type":"ContainerStarted","Data":"3fc15723a16e9c86de8d78bd114db1d18218de38f6c39d3186e8fc446d7e168a"}
Aug 03 23:41:19.333815 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:19.333762    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-pwks2" podStartSLOduration=1.751618356 podStartE2EDuration="6.333743189s" totalImagesPullingTime="4.582124833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:41:19.332845813 +0000 UTC m=+687.034552476" watchObservedRunningTime="2026-08-03 23:41:19.333743189 +0000 UTC m=+687.035449853"
Aug 03 23:41:55.185556 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.185522    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-27lwm"]
Aug 03 23:41:55.187838 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.187814    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:41:55.190347 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.190327    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 03 23:41:55.195820 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.195800    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-27lwm"]
Aug 03 23:41:55.285814 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.285780    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-27lwm"]
Aug 03 23:41:55.369034 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.369006    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwvcm\" (UniqueName: \"kubernetes.io/projected/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-kube-api-access-gwvcm\") pod \"limitador-limitador-859c4d96d6-27lwm\" (UID: \"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:41:55.369163 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.369083    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-config-file\") pod \"limitador-limitador-859c4d96d6-27lwm\" (UID: \"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:41:55.470041 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.469986    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-config-file\") pod \"limitador-limitador-859c4d96d6-27lwm\" (UID: \"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:41:55.470041 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.470031    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwvcm\" (UniqueName: \"kubernetes.io/projected/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-kube-api-access-gwvcm\") pod \"limitador-limitador-859c4d96d6-27lwm\" (UID: \"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:41:55.470553 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.470536    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-config-file\") pod \"limitador-limitador-859c4d96d6-27lwm\" (UID: \"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:41:55.478132 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.478108    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwvcm\" (UniqueName: \"kubernetes.io/projected/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-kube-api-access-gwvcm\") pod \"limitador-limitador-859c4d96d6-27lwm\" (UID: \"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:41:55.498124 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.498107    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:41:55.622391 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:55.622370    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-27lwm"]
Aug 03 23:41:55.624445 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:41:55.624418    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00cdde4_9653_4b8f_9f7b_c8c3d11d0f55.slice/crio-945dcf62d46ef29ef5ab061c7f0d4f53f6e052d23e04cbb533d1c150899cf05b WatchSource:0}: Error finding container 945dcf62d46ef29ef5ab061c7f0d4f53f6e052d23e04cbb533d1c150899cf05b: Status 404 returned error can't find the container with id 945dcf62d46ef29ef5ab061c7f0d4f53f6e052d23e04cbb533d1c150899cf05b
Aug 03 23:41:56.136485 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:56.136452    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-zjjsp"]
Aug 03 23:41:56.139122 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:56.139104    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-zjjsp"
Aug 03 23:41:56.141608 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:56.141589    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-w24fk\""
Aug 03 23:41:56.145137 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:56.145119    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-zjjsp"]
Aug 03 23:41:56.277037 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:56.277010    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q78mv\" (UniqueName: \"kubernetes.io/projected/c3ff6e99-320a-4240-9220-cf0cefdb7425-kube-api-access-q78mv\") pod \"authorino-c548b5d7d-zjjsp\" (UID: \"c3ff6e99-320a-4240-9220-cf0cefdb7425\") " pod="kuadrant-system/authorino-c548b5d7d-zjjsp"
Aug 03 23:41:56.377952 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:56.377928    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q78mv\" (UniqueName: \"kubernetes.io/projected/c3ff6e99-320a-4240-9220-cf0cefdb7425-kube-api-access-q78mv\") pod \"authorino-c548b5d7d-zjjsp\" (UID: \"c3ff6e99-320a-4240-9220-cf0cefdb7425\") " pod="kuadrant-system/authorino-c548b5d7d-zjjsp"
Aug 03 23:41:56.386570 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:56.386514    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q78mv\" (UniqueName: \"kubernetes.io/projected/c3ff6e99-320a-4240-9220-cf0cefdb7425-kube-api-access-q78mv\") pod \"authorino-c548b5d7d-zjjsp\" (UID: \"c3ff6e99-320a-4240-9220-cf0cefdb7425\") " pod="kuadrant-system/authorino-c548b5d7d-zjjsp"
Aug 03 23:41:56.449108 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:56.449087    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-zjjsp"
Aug 03 23:41:56.482799 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:56.482770    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm" event={"ID":"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55","Type":"ContainerStarted","Data":"945dcf62d46ef29ef5ab061c7f0d4f53f6e052d23e04cbb533d1c150899cf05b"}
Aug 03 23:41:56.593794 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:56.593762    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-zjjsp"]
Aug 03 23:41:56.596498 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:41:56.596475    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3ff6e99_320a_4240_9220_cf0cefdb7425.slice/crio-c5c2ce2e3ac2676e842067ddbda9717127af186791d645720cadad5c6f57bc72 WatchSource:0}: Error finding container c5c2ce2e3ac2676e842067ddbda9717127af186791d645720cadad5c6f57bc72: Status 404 returned error can't find the container with id c5c2ce2e3ac2676e842067ddbda9717127af186791d645720cadad5c6f57bc72
Aug 03 23:41:57.491387 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:57.491270    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-zjjsp" event={"ID":"c3ff6e99-320a-4240-9220-cf0cefdb7425","Type":"ContainerStarted","Data":"c5c2ce2e3ac2676e842067ddbda9717127af186791d645720cadad5c6f57bc72"}
Aug 03 23:41:59.503226 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:59.503194    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm" event={"ID":"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55","Type":"ContainerStarted","Data":"498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983"}
Aug 03 23:41:59.503671 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:59.503288    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:41:59.504652 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:59.504632    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-zjjsp" event={"ID":"c3ff6e99-320a-4240-9220-cf0cefdb7425","Type":"ContainerStarted","Data":"7d00d59934ddc1ca5a5031cb5f50c82b991537c12c970b094ca67105279945d9"}
Aug 03 23:41:59.519137 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:59.519093    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm" podStartSLOduration=1.294716692 podStartE2EDuration="4.519079481s" totalImagesPullingTime="3.224362789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:41:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:41:59.51726603 +0000 UTC m=+727.218972683" watchObservedRunningTime="2026-08-03 23:41:59.519079481 +0000 UTC m=+727.220786144"
Aug 03 23:41:59.534325 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:41:59.534288    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-zjjsp" podStartSLOduration=1.253749446 podStartE2EDuration="3.534274101s" totalImagesPullingTime="2.280524655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:41:59.532322862 +0000 UTC m=+727.234029524" watchObservedRunningTime="2026-08-03 23:41:59.534274101 +0000 UTC m=+727.235980763"
Aug 03 23:42:09.285520 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:09.285487    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-27lwm"]
Aug 03 23:42:09.286016 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:09.285695    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm" podUID="e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55" containerName="limitador" containerID="cri-o://498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983" gracePeriod=30
Aug 03 23:42:09.286433 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:09.286404    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:42:09.830871 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:09.830850    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:42:09.887541 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:09.887483    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-config-file\") pod \"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55\" (UID: \"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55\") "
Aug 03 23:42:09.887541 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:09.887514    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwvcm\" (UniqueName: \"kubernetes.io/projected/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-kube-api-access-gwvcm\") pod \"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55\" (UID: \"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55\") "
Aug 03 23:42:09.887831 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:09.887809    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-config-file" (OuterVolumeSpecName: "config-file") pod "e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55" (UID: "e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 23:42:09.889746 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:09.889723    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-kube-api-access-gwvcm" (OuterVolumeSpecName: "kube-api-access-gwvcm") pod "e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55" (UID: "e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55"). InnerVolumeSpecName "kube-api-access-gwvcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:42:09.988267 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:09.988244    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwvcm\" (UniqueName: \"kubernetes.io/projected/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-kube-api-access-gwvcm\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:42:09.988267 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:09.988267    2553 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55-config-file\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:42:10.550965 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:10.550935    2553 generic.go:356] "Generic (PLEG): container finished" podID="e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55" containerID="498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983" exitCode=0
Aug 03 23:42:10.551373 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:10.550979    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm"
Aug 03 23:42:10.551373 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:10.550996    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm" event={"ID":"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55","Type":"ContainerDied","Data":"498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983"}
Aug 03 23:42:10.551373 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:10.551034    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-27lwm" event={"ID":"e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55","Type":"ContainerDied","Data":"945dcf62d46ef29ef5ab061c7f0d4f53f6e052d23e04cbb533d1c150899cf05b"}
Aug 03 23:42:10.551373 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:10.551055    2553 scope.go:117] "RemoveContainer" containerID="498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983"
Aug 03 23:42:10.559656 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:10.559638    2553 scope.go:117] "RemoveContainer" containerID="498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983"
Aug 03 23:42:10.559920 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:42:10.559903    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983\": container with ID starting with 498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983 not found: ID does not exist" containerID="498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983"
Aug 03 23:42:10.559973 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:10.559930    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983"} err="failed to get container status \"498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983\": rpc error: code = NotFound desc = could not find container \"498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983\": container with ID starting with 498fc76ef3ea5512794ce4586fd4e768d9381e8bd6065cfcbbbb55fd70772983 not found: ID does not exist"
Aug 03 23:42:10.571795 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:10.571756    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-27lwm"]
Aug 03 23:42:10.575321 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:10.575299    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-27lwm"]
Aug 03 23:42:10.764293 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:10.764267    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55" path="/var/lib/kubelet/pods/e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55/volumes"
Aug 03 23:42:25.402428 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:25.402396    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-zjjsp"]
Aug 03 23:42:25.402820 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:25.402559    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-zjjsp" podUID="c3ff6e99-320a-4240-9220-cf0cefdb7425" containerName="authorino" containerID="cri-o://7d00d59934ddc1ca5a5031cb5f50c82b991537c12c970b094ca67105279945d9" gracePeriod=30
Aug 03 23:42:25.619387 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:25.619360    2553 generic.go:356] "Generic (PLEG): container finished" podID="c3ff6e99-320a-4240-9220-cf0cefdb7425" containerID="7d00d59934ddc1ca5a5031cb5f50c82b991537c12c970b094ca67105279945d9" exitCode=0
Aug 03 23:42:25.619524 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:25.619413    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-zjjsp" event={"ID":"c3ff6e99-320a-4240-9220-cf0cefdb7425","Type":"ContainerDied","Data":"7d00d59934ddc1ca5a5031cb5f50c82b991537c12c970b094ca67105279945d9"}
Aug 03 23:42:25.644439 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:25.644417    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-zjjsp"
Aug 03 23:42:25.712317 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:25.712297    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q78mv\" (UniqueName: \"kubernetes.io/projected/c3ff6e99-320a-4240-9220-cf0cefdb7425-kube-api-access-q78mv\") pod \"c3ff6e99-320a-4240-9220-cf0cefdb7425\" (UID: \"c3ff6e99-320a-4240-9220-cf0cefdb7425\") "
Aug 03 23:42:25.714282 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:25.714255    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3ff6e99-320a-4240-9220-cf0cefdb7425-kube-api-access-q78mv" (OuterVolumeSpecName: "kube-api-access-q78mv") pod "c3ff6e99-320a-4240-9220-cf0cefdb7425" (UID: "c3ff6e99-320a-4240-9220-cf0cefdb7425"). InnerVolumeSpecName "kube-api-access-q78mv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:42:25.813550 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:25.813527    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q78mv\" (UniqueName: \"kubernetes.io/projected/c3ff6e99-320a-4240-9220-cf0cefdb7425-kube-api-access-q78mv\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:42:26.625286 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:26.625254    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-zjjsp" event={"ID":"c3ff6e99-320a-4240-9220-cf0cefdb7425","Type":"ContainerDied","Data":"c5c2ce2e3ac2676e842067ddbda9717127af186791d645720cadad5c6f57bc72"}
Aug 03 23:42:26.625702 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:26.625264    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-zjjsp"
Aug 03 23:42:26.625702 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:26.625300    2553 scope.go:117] "RemoveContainer" containerID="7d00d59934ddc1ca5a5031cb5f50c82b991537c12c970b094ca67105279945d9"
Aug 03 23:42:26.646172 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:26.646144    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-zjjsp"]
Aug 03 23:42:26.648577 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:26.648556    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-zjjsp"]
Aug 03 23:42:26.763207 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:26.763182    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3ff6e99-320a-4240-9220-cf0cefdb7425" path="/var/lib/kubelet/pods/c3ff6e99-320a-4240-9220-cf0cefdb7425/volumes"
Aug 03 23:42:29.936631 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.936596    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74fb9dc8d7-zr8qk"]
Aug 03 23:42:29.937195 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.937175    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3ff6e99-320a-4240-9220-cf0cefdb7425" containerName="authorino"
Aug 03 23:42:29.937271 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.937198    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3ff6e99-320a-4240-9220-cf0cefdb7425" containerName="authorino"
Aug 03 23:42:29.937271 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.937211    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55" containerName="limitador"
Aug 03 23:42:29.937271 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.937220    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55" containerName="limitador"
Aug 03 23:42:29.937377 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.937301    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="e00cdde4-9653-4b8f-9f7b-c8c3d11d0f55" containerName="limitador"
Aug 03 23:42:29.937377 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.937312    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3ff6e99-320a-4240-9220-cf0cefdb7425" containerName="authorino"
Aug 03 23:42:29.940947 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.940925    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:29.943716 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.943685    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 03 23:42:29.943833 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.943716    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 03 23:42:29.943900 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.943850    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wwmmf\""
Aug 03 23:42:29.949129 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:29.949107    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74fb9dc8d7-zr8qk"]
Aug 03 23:42:30.049374 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.049349    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0f85a57-1161-4d90-9999-1a109bad4ab7-webhook-cert\") pod \"maas-controller-74fb9dc8d7-zr8qk\" (UID: \"a0f85a57-1161-4d90-9999-1a109bad4ab7\") " pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:30.049527 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.049389    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59sv4\" (UniqueName: \"kubernetes.io/projected/a0f85a57-1161-4d90-9999-1a109bad4ab7-kube-api-access-59sv4\") pod \"maas-controller-74fb9dc8d7-zr8qk\" (UID: \"a0f85a57-1161-4d90-9999-1a109bad4ab7\") " pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:30.093633 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.093610    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-96bff6bb9-7776h"]
Aug 03 23:42:30.095646 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.095632    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:42:30.104958 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.104938    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-7776h"]
Aug 03 23:42:30.150218 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.150194    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0f85a57-1161-4d90-9999-1a109bad4ab7-webhook-cert\") pod \"maas-controller-74fb9dc8d7-zr8qk\" (UID: \"a0f85a57-1161-4d90-9999-1a109bad4ab7\") " pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:30.150301 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.150232    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59sv4\" (UniqueName: \"kubernetes.io/projected/a0f85a57-1161-4d90-9999-1a109bad4ab7-kube-api-access-59sv4\") pod \"maas-controller-74fb9dc8d7-zr8qk\" (UID: \"a0f85a57-1161-4d90-9999-1a109bad4ab7\") " pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:30.150346 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:42:30.150315    2553 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Aug 03 23:42:30.150384 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:42:30.150363    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0f85a57-1161-4d90-9999-1a109bad4ab7-webhook-cert podName:a0f85a57-1161-4d90-9999-1a109bad4ab7 nodeName:}" failed. No retries permitted until 2026-08-03 23:42:30.650349711 +0000 UTC m=+758.352056352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/a0f85a57-1161-4d90-9999-1a109bad4ab7-webhook-cert") pod "maas-controller-74fb9dc8d7-zr8qk" (UID: "a0f85a57-1161-4d90-9999-1a109bad4ab7") : secret "maas-controller-webhook-cert" not found
Aug 03 23:42:30.158533 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.158514    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59sv4\" (UniqueName: \"kubernetes.io/projected/a0f85a57-1161-4d90-9999-1a109bad4ab7-kube-api-access-59sv4\") pod \"maas-controller-74fb9dc8d7-zr8qk\" (UID: \"a0f85a57-1161-4d90-9999-1a109bad4ab7\") " pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:30.251268 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.251222    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-webhook-cert\") pod \"maas-controller-96bff6bb9-7776h\" (UID: \"74fa132a-b2b2-4781-aefd-ecdf7c89d87c\") " pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:42:30.251360 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.251272    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7fl2\" (UniqueName: \"kubernetes.io/projected/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-kube-api-access-h7fl2\") pod \"maas-controller-96bff6bb9-7776h\" (UID: \"74fa132a-b2b2-4781-aefd-ecdf7c89d87c\") " pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:42:30.352105 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.352076    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7fl2\" (UniqueName: \"kubernetes.io/projected/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-kube-api-access-h7fl2\") pod \"maas-controller-96bff6bb9-7776h\" (UID: \"74fa132a-b2b2-4781-aefd-ecdf7c89d87c\") " pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:42:30.352230 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.352184    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-webhook-cert\") pod \"maas-controller-96bff6bb9-7776h\" (UID: \"74fa132a-b2b2-4781-aefd-ecdf7c89d87c\") " pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:42:30.352322 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:42:30.352305    2553 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Aug 03 23:42:30.352378 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:42:30.352366    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-webhook-cert podName:74fa132a-b2b2-4781-aefd-ecdf7c89d87c nodeName:}" failed. No retries permitted until 2026-08-03 23:42:30.852346699 +0000 UTC m=+758.554053340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-webhook-cert") pod "maas-controller-96bff6bb9-7776h" (UID: "74fa132a-b2b2-4781-aefd-ecdf7c89d87c") : secret "maas-controller-webhook-cert" not found
Aug 03 23:42:30.362418 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.362389    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7fl2\" (UniqueName: \"kubernetes.io/projected/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-kube-api-access-h7fl2\") pod \"maas-controller-96bff6bb9-7776h\" (UID: \"74fa132a-b2b2-4781-aefd-ecdf7c89d87c\") " pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:42:30.655549 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.655462    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0f85a57-1161-4d90-9999-1a109bad4ab7-webhook-cert\") pod \"maas-controller-74fb9dc8d7-zr8qk\" (UID: \"a0f85a57-1161-4d90-9999-1a109bad4ab7\") " pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:30.657904 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.657884    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0f85a57-1161-4d90-9999-1a109bad4ab7-webhook-cert\") pod \"maas-controller-74fb9dc8d7-zr8qk\" (UID: \"a0f85a57-1161-4d90-9999-1a109bad4ab7\") " pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:30.852169 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.852139    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:30.858061 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.858039    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-webhook-cert\") pod \"maas-controller-96bff6bb9-7776h\" (UID: \"74fa132a-b2b2-4781-aefd-ecdf7c89d87c\") " pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:42:30.860323 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.860302    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-webhook-cert\") pod \"maas-controller-96bff6bb9-7776h\" (UID: \"74fa132a-b2b2-4781-aefd-ecdf7c89d87c\") " pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:42:30.977599 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:30.977577    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74fb9dc8d7-zr8qk"]
Aug 03 23:42:30.979748 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:42:30.979695    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f85a57_1161_4d90_9999_1a109bad4ab7.slice/crio-a86ed8fa455b43e7e3656277eda1c62f0fec3adfb0f70e1de25b1244679262f0 WatchSource:0}: Error finding container a86ed8fa455b43e7e3656277eda1c62f0fec3adfb0f70e1de25b1244679262f0: Status 404 returned error can't find the container with id a86ed8fa455b43e7e3656277eda1c62f0fec3adfb0f70e1de25b1244679262f0
Aug 03 23:42:31.005256 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:31.005227    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:42:31.129433 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:31.129402    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-7776h"]
Aug 03 23:42:31.132770 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:42:31.132745    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74fa132a_b2b2_4781_aefd_ecdf7c89d87c.slice/crio-cd2d993e90ebca6735014e891b1d3448308c82dca2db8dc4d66fab39f8cd2a6f WatchSource:0}: Error finding container cd2d993e90ebca6735014e891b1d3448308c82dca2db8dc4d66fab39f8cd2a6f: Status 404 returned error can't find the container with id cd2d993e90ebca6735014e891b1d3448308c82dca2db8dc4d66fab39f8cd2a6f
Aug 03 23:42:31.653468 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:31.653420    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk" event={"ID":"a0f85a57-1161-4d90-9999-1a109bad4ab7","Type":"ContainerStarted","Data":"a86ed8fa455b43e7e3656277eda1c62f0fec3adfb0f70e1de25b1244679262f0"}
Aug 03 23:42:31.655701 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:31.655664    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-96bff6bb9-7776h" event={"ID":"74fa132a-b2b2-4781-aefd-ecdf7c89d87c","Type":"ContainerStarted","Data":"cd2d993e90ebca6735014e891b1d3448308c82dca2db8dc4d66fab39f8cd2a6f"}
Aug 03 23:42:33.668328 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:33.668292    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk" event={"ID":"a0f85a57-1161-4d90-9999-1a109bad4ab7","Type":"ContainerStarted","Data":"76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978"}
Aug 03 23:42:33.668809 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:33.668395    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:33.669888 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:33.669862    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-96bff6bb9-7776h" event={"ID":"74fa132a-b2b2-4781-aefd-ecdf7c89d87c","Type":"ContainerStarted","Data":"732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f"}
Aug 03 23:42:33.670005 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:33.669948    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:42:33.691384 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:33.691343    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk" podStartSLOduration=2.206606601 podStartE2EDuration="4.691331094s" totalImagesPullingTime="2.484724493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:42:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:42:33.688763838 +0000 UTC m=+761.390470503" watchObservedRunningTime="2026-08-03 23:42:33.691331094 +0000 UTC m=+761.393037756"
Aug 03 23:42:33.706622 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:33.706576    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-96bff6bb9-7776h" podStartSLOduration=1.380614276 podStartE2EDuration="3.706563719s" totalImagesPullingTime="2.325949443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:42:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:42:33.705045767 +0000 UTC m=+761.406752431" watchObservedRunningTime="2026-08-03 23:42:33.706563719 +0000 UTC m=+761.408270382"
Aug 03 23:42:44.675651 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:44.675624    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:44.676069 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:44.675721    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:42:44.738371 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:44.738322    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74fb9dc8d7-zr8qk"]
Aug 03 23:42:44.738601 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:44.738560    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk" podUID="a0f85a57-1161-4d90-9999-1a109bad4ab7" containerName="manager" containerID="cri-o://76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978" gracePeriod=30
Aug 03 23:42:44.975070 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:44.975046    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:45.075951 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.075925    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0f85a57-1161-4d90-9999-1a109bad4ab7-webhook-cert\") pod \"a0f85a57-1161-4d90-9999-1a109bad4ab7\" (UID: \"a0f85a57-1161-4d90-9999-1a109bad4ab7\") "
Aug 03 23:42:45.076097 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.075974    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59sv4\" (UniqueName: \"kubernetes.io/projected/a0f85a57-1161-4d90-9999-1a109bad4ab7-kube-api-access-59sv4\") pod \"a0f85a57-1161-4d90-9999-1a109bad4ab7\" (UID: \"a0f85a57-1161-4d90-9999-1a109bad4ab7\") "
Aug 03 23:42:45.078065 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.078040    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0f85a57-1161-4d90-9999-1a109bad4ab7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a0f85a57-1161-4d90-9999-1a109bad4ab7" (UID: "a0f85a57-1161-4d90-9999-1a109bad4ab7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:42:45.078171 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.078148    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0f85a57-1161-4d90-9999-1a109bad4ab7-kube-api-access-59sv4" (OuterVolumeSpecName: "kube-api-access-59sv4") pod "a0f85a57-1161-4d90-9999-1a109bad4ab7" (UID: "a0f85a57-1161-4d90-9999-1a109bad4ab7"). InnerVolumeSpecName "kube-api-access-59sv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:42:45.177488 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.177470    2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0f85a57-1161-4d90-9999-1a109bad4ab7-webhook-cert\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:42:45.177488 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.177490    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59sv4\" (UniqueName: \"kubernetes.io/projected/a0f85a57-1161-4d90-9999-1a109bad4ab7-kube-api-access-59sv4\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:42:45.722527 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.722500    2553 generic.go:356] "Generic (PLEG): container finished" podID="a0f85a57-1161-4d90-9999-1a109bad4ab7" containerID="76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978" exitCode=0
Aug 03 23:42:45.722881 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.722543    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk"
Aug 03 23:42:45.722881 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.722549    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk" event={"ID":"a0f85a57-1161-4d90-9999-1a109bad4ab7","Type":"ContainerDied","Data":"76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978"}
Aug 03 23:42:45.722881 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.722589    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fb9dc8d7-zr8qk" event={"ID":"a0f85a57-1161-4d90-9999-1a109bad4ab7","Type":"ContainerDied","Data":"a86ed8fa455b43e7e3656277eda1c62f0fec3adfb0f70e1de25b1244679262f0"}
Aug 03 23:42:45.722881 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.722610    2553 scope.go:117] "RemoveContainer" containerID="76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978"
Aug 03 23:42:45.731402 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.731386    2553 scope.go:117] "RemoveContainer" containerID="76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978"
Aug 03 23:42:45.731634 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:42:45.731614    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978\": container with ID starting with 76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978 not found: ID does not exist" containerID="76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978"
Aug 03 23:42:45.731682 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.731643    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978"} err="failed to get container status \"76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978\": rpc error: code = NotFound desc = could not find container \"76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978\": container with ID starting with 76f2463561200414c56dd79ee97eef863d8d36ca21c535a0bce220e5be325978 not found: ID does not exist"
Aug 03 23:42:45.742108 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.742088    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74fb9dc8d7-zr8qk"]
Aug 03 23:42:45.745756 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:45.745736    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74fb9dc8d7-zr8qk"]
Aug 03 23:42:46.763362 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:42:46.763333    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0f85a57-1161-4d90-9999-1a109bad4ab7" path="/var/lib/kubelet/pods/a0f85a57-1161-4d90-9999-1a109bad4ab7/volumes"
Aug 03 23:43:04.642365 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.642331    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"]
Aug 03 23:43:04.642752 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.642642    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0f85a57-1161-4d90-9999-1a109bad4ab7" containerName="manager"
Aug 03 23:43:04.642752 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.642652    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0f85a57-1161-4d90-9999-1a109bad4ab7" containerName="manager"
Aug 03 23:43:04.642752 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.642747    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0f85a57-1161-4d90-9999-1a109bad4ab7" containerName="manager"
Aug 03 23:43:04.646975 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.646958    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:04.649797 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.649758    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 03 23:43:04.649914 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.649762    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Aug 03 23:43:04.649914 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.649897    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Aug 03 23:43:04.650931 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.650917    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pzz5v\""
Aug 03 23:43:04.656802 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.656636    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"]
Aug 03 23:43:04.725010 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.724987    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38fab0b3-ea17-4bef-9d0c-a8515fc266e4-maas-api-tls\") pod \"maas-api-69bc574f56-4s4ld\" (UID: \"38fab0b3-ea17-4bef-9d0c-a8515fc266e4\") " pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:04.725109 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.725071    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38fab0b3-ea17-4bef-9d0c-a8515fc266e4-openshift-service-ca\") pod \"maas-api-69bc574f56-4s4ld\" (UID: \"38fab0b3-ea17-4bef-9d0c-a8515fc266e4\") " pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:04.725149 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.725113    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kkbb\" (UniqueName: \"kubernetes.io/projected/38fab0b3-ea17-4bef-9d0c-a8515fc266e4-kube-api-access-4kkbb\") pod \"maas-api-69bc574f56-4s4ld\" (UID: \"38fab0b3-ea17-4bef-9d0c-a8515fc266e4\") " pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:04.826215 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.826189    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38fab0b3-ea17-4bef-9d0c-a8515fc266e4-openshift-service-ca\") pod \"maas-api-69bc574f56-4s4ld\" (UID: \"38fab0b3-ea17-4bef-9d0c-a8515fc266e4\") " pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:04.826326 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.826249    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kkbb\" (UniqueName: \"kubernetes.io/projected/38fab0b3-ea17-4bef-9d0c-a8515fc266e4-kube-api-access-4kkbb\") pod \"maas-api-69bc574f56-4s4ld\" (UID: \"38fab0b3-ea17-4bef-9d0c-a8515fc266e4\") " pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:04.826326 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.826269    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38fab0b3-ea17-4bef-9d0c-a8515fc266e4-maas-api-tls\") pod \"maas-api-69bc574f56-4s4ld\" (UID: \"38fab0b3-ea17-4bef-9d0c-a8515fc266e4\") " pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:04.826902 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.826883    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38fab0b3-ea17-4bef-9d0c-a8515fc266e4-openshift-service-ca\") pod \"maas-api-69bc574f56-4s4ld\" (UID: \"38fab0b3-ea17-4bef-9d0c-a8515fc266e4\") " pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:04.828771 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.828745    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38fab0b3-ea17-4bef-9d0c-a8515fc266e4-maas-api-tls\") pod \"maas-api-69bc574f56-4s4ld\" (UID: \"38fab0b3-ea17-4bef-9d0c-a8515fc266e4\") " pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:04.834743 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.834723    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kkbb\" (UniqueName: \"kubernetes.io/projected/38fab0b3-ea17-4bef-9d0c-a8515fc266e4-kube-api-access-4kkbb\") pod \"maas-api-69bc574f56-4s4ld\" (UID: \"38fab0b3-ea17-4bef-9d0c-a8515fc266e4\") " pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:04.956977 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:04.956952    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:05.091500 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.091473    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"]
Aug 03 23:43:05.093814 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:43:05.093786    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38fab0b3_ea17_4bef_9d0c_a8515fc266e4.slice/crio-f4b9a4f8f4bdfa79db5aea3ce000216e41e47e4e0262783cc5e3441d0c411215 WatchSource:0}: Error finding container f4b9a4f8f4bdfa79db5aea3ce000216e41e47e4e0262783cc5e3441d0c411215: Status 404 returned error can't find the container with id f4b9a4f8f4bdfa79db5aea3ce000216e41e47e4e0262783cc5e3441d0c411215
Aug 03 23:43:05.149325 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.149303    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"]
Aug 03 23:43:05.152219 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.152201    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"
Aug 03 23:43:05.154698 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.154682    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Aug 03 23:43:05.163023 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.163004    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"]
Aug 03 23:43:05.182761 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.182736    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-nsb5x"]
Aug 03 23:43:05.184928 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.184913    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x"
Aug 03 23:43:05.187363 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.187346    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zj4hx\""
Aug 03 23:43:05.196877 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.196857    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-nsb5x"]
Aug 03 23:43:05.230270 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.230219    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51e56f31-11d1-4d11-8d7f-3193495da2e0-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-nsb5x\" (UID: \"51e56f31-11d1-4d11-8d7f-3193495da2e0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x"
Aug 03 23:43:05.230270 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.230264    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2864afa1-5450-409d-b876-f85b6ed727a0-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-95kbd\" (UID: \"2864afa1-5450-409d-b876-f85b6ed727a0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"
Aug 03 23:43:05.230399 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.230327    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsgs6\" (UniqueName: \"kubernetes.io/projected/2864afa1-5450-409d-b876-f85b6ed727a0-kube-api-access-jsgs6\") pod \"payload-pre-processing-687dbb6cf6-95kbd\" (UID: \"2864afa1-5450-409d-b876-f85b6ed727a0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"
Aug 03 23:43:05.230399 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.230385    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4klp\" (UniqueName: \"kubernetes.io/projected/51e56f31-11d1-4d11-8d7f-3193495da2e0-kube-api-access-h4klp\") pod \"payload-processing-5cc6cd444b-nsb5x\" (UID: \"51e56f31-11d1-4d11-8d7f-3193495da2e0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x"
Aug 03 23:43:05.330856 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.330822    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2864afa1-5450-409d-b876-f85b6ed727a0-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-95kbd\" (UID: \"2864afa1-5450-409d-b876-f85b6ed727a0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"
Aug 03 23:43:05.331045 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.330883    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsgs6\" (UniqueName: \"kubernetes.io/projected/2864afa1-5450-409d-b876-f85b6ed727a0-kube-api-access-jsgs6\") pod \"payload-pre-processing-687dbb6cf6-95kbd\" (UID: \"2864afa1-5450-409d-b876-f85b6ed727a0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"
Aug 03 23:43:05.331045 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.330936    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4klp\" (UniqueName: \"kubernetes.io/projected/51e56f31-11d1-4d11-8d7f-3193495da2e0-kube-api-access-h4klp\") pod \"payload-processing-5cc6cd444b-nsb5x\" (UID: \"51e56f31-11d1-4d11-8d7f-3193495da2e0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x"
Aug 03 23:43:05.331045 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.330973    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51e56f31-11d1-4d11-8d7f-3193495da2e0-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-nsb5x\" (UID: \"51e56f31-11d1-4d11-8d7f-3193495da2e0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x"
Aug 03 23:43:05.331616 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.331593    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51e56f31-11d1-4d11-8d7f-3193495da2e0-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-nsb5x\" (UID: \"51e56f31-11d1-4d11-8d7f-3193495da2e0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x"
Aug 03 23:43:05.331616 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.331609    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2864afa1-5450-409d-b876-f85b6ed727a0-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-95kbd\" (UID: \"2864afa1-5450-409d-b876-f85b6ed727a0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"
Aug 03 23:43:05.339105 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.339068    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4klp\" (UniqueName: \"kubernetes.io/projected/51e56f31-11d1-4d11-8d7f-3193495da2e0-kube-api-access-h4klp\") pod \"payload-processing-5cc6cd444b-nsb5x\" (UID: \"51e56f31-11d1-4d11-8d7f-3193495da2e0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x"
Aug 03 23:43:05.339278 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.339257    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsgs6\" (UniqueName: \"kubernetes.io/projected/2864afa1-5450-409d-b876-f85b6ed727a0-kube-api-access-jsgs6\") pod \"payload-pre-processing-687dbb6cf6-95kbd\" (UID: \"2864afa1-5450-409d-b876-f85b6ed727a0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"
Aug 03 23:43:05.463498 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.463476    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"
Aug 03 23:43:05.493990 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.493932    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x"
Aug 03 23:43:05.599161 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.599130    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"]
Aug 03 23:43:05.604426 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:43:05.604403    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2864afa1_5450_409d_b876_f85b6ed727a0.slice/crio-c4314e5919912be37de6e7df3aeda2af3cd17ae232be256667aa6d7b613e5380 WatchSource:0}: Error finding container c4314e5919912be37de6e7df3aeda2af3cd17ae232be256667aa6d7b613e5380: Status 404 returned error can't find the container with id c4314e5919912be37de6e7df3aeda2af3cd17ae232be256667aa6d7b613e5380
Aug 03 23:43:05.633419 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.633398    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-nsb5x"]
Aug 03 23:43:05.635185 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:43:05.635163    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e56f31_11d1_4d11_8d7f_3193495da2e0.slice/crio-4dc6f812a75e29d79c2debbfcdb0898a4bda2d1c76364ff07cb526a1c2a90550 WatchSource:0}: Error finding container 4dc6f812a75e29d79c2debbfcdb0898a4bda2d1c76364ff07cb526a1c2a90550: Status 404 returned error can't find the container with id 4dc6f812a75e29d79c2debbfcdb0898a4bda2d1c76364ff07cb526a1c2a90550
Aug 03 23:43:05.807242 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.807157    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x" event={"ID":"51e56f31-11d1-4d11-8d7f-3193495da2e0","Type":"ContainerStarted","Data":"4dc6f812a75e29d79c2debbfcdb0898a4bda2d1c76364ff07cb526a1c2a90550"}
Aug 03 23:43:05.808323 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.808299    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd" event={"ID":"2864afa1-5450-409d-b876-f85b6ed727a0","Type":"ContainerStarted","Data":"c4314e5919912be37de6e7df3aeda2af3cd17ae232be256667aa6d7b613e5380"}
Aug 03 23:43:05.809311 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:05.809291    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld" event={"ID":"38fab0b3-ea17-4bef-9d0c-a8515fc266e4","Type":"ContainerStarted","Data":"f4b9a4f8f4bdfa79db5aea3ce000216e41e47e4e0262783cc5e3441d0c411215"}
Aug 03 23:43:07.823912 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:07.823880    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld" event={"ID":"38fab0b3-ea17-4bef-9d0c-a8515fc266e4","Type":"ContainerStarted","Data":"a168e58d924bf75997a151ae55bb220a1abbe69f7b91d13ff0f07b35bb4f425e"}
Aug 03 23:43:07.824354 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:07.823976    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:07.839491 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:07.839424    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld" podStartSLOduration=1.9572273930000001 podStartE2EDuration="3.839409843s" totalImagesPullingTime="1.88218245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:43:07.838285848 +0000 UTC m=+795.539992511" watchObservedRunningTime="2026-08-03 23:43:07.839409843 +0000 UTC m=+795.541116506"
Aug 03 23:43:08.829341 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:08.829311    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x" event={"ID":"51e56f31-11d1-4d11-8d7f-3193495da2e0","Type":"ContainerStarted","Data":"532a6921755b0f034bdba03ef17003a3bffd8cd43289d0d0e8f2ce7beddce00b"}
Aug 03 23:43:08.829791 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:08.829406    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x"
Aug 03 23:43:08.830825 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:08.830805    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd" event={"ID":"2864afa1-5450-409d-b876-f85b6ed727a0","Type":"ContainerStarted","Data":"c60b14de776f6f8c6d804e1a839a3722a9dfd396a21d96e75a82fe7ddaa44aaa"}
Aug 03 23:43:08.830973 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:08.830959    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"
Aug 03 23:43:08.846652 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:08.846610    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x" podStartSLOduration=1.355385931 podStartE2EDuration="3.846599068s" totalImagesPullingTime="2.491213137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:43:08.843960403 +0000 UTC m=+796.545667090" watchObservedRunningTime="2026-08-03 23:43:08.846599068 +0000 UTC m=+796.548305727"
Aug 03 23:43:08.858801 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:08.858760    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd" podStartSLOduration=1.342570937 podStartE2EDuration="3.858748428s" totalImagesPullingTime="2.516177491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:43:08.857591827 +0000 UTC m=+796.559298490" watchObservedRunningTime="2026-08-03 23:43:08.858748428 +0000 UTC m=+796.560455109"
Aug 03 23:43:13.836406 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:13.836328    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-69bc574f56-4s4ld"
Aug 03 23:43:19.838408 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:19.838384    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-95kbd"
Aug 03 23:43:19.838841 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:19.838479    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-nsb5x"
Aug 03 23:43:36.443403 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.443371    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"]
Aug 03 23:43:36.446078 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.446057    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.448856 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.448814    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 03 23:43:36.448856 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.448822    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 03 23:43:36.450240 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.450219    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Aug 03 23:43:36.450358 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.450341    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jsgb6\""
Aug 03 23:43:36.456476 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.456456    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"]
Aug 03 23:43:36.587896 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.587875    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.588006 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.587903    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.588006 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.587924    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.588076 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.588017    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.588112 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.588078    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg7bq\" (UniqueName: \"kubernetes.io/projected/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-kube-api-access-rg7bq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.588112 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.588103    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.688893 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.688871    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg7bq\" (UniqueName: \"kubernetes.io/projected/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-kube-api-access-rg7bq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.688984 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.688899    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.688984 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.688945    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.688984 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.688962    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.688984 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.688980    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.689159 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.689031    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.689403 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.689382    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.689470 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.689400    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.689470 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.689439    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.691316 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.691296    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.691506 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.691490    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.697243 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.697191    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg7bq\" (UniqueName: \"kubernetes.io/projected/fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4-kube-api-access-rg7bq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29lnb\" (UID: \"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.757223 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.757207    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:36.884976 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.884953    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"]
Aug 03 23:43:36.887181 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:43:36.887156    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb4c8781_6f5a_4d6f_87bd_c030fd1c80d4.slice/crio-c4fc3c2e5746be06810ac080f45bbadb1161e1ae0000d175915004f1d194814c WatchSource:0}: Error finding container c4fc3c2e5746be06810ac080f45bbadb1161e1ae0000d175915004f1d194814c: Status 404 returned error can't find the container with id c4fc3c2e5746be06810ac080f45bbadb1161e1ae0000d175915004f1d194814c
Aug 03 23:43:36.953523 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:36.953456    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb" event={"ID":"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4","Type":"ContainerStarted","Data":"c4fc3c2e5746be06810ac080f45bbadb1161e1ae0000d175915004f1d194814c"}
Aug 03 23:43:37.960789 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:37.960755    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb" event={"ID":"fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4","Type":"ContainerStarted","Data":"7144c45896a8e324544da6f6fa0fcf2101db16d3839717b6d79df5fb7c03de06"}
Aug 03 23:43:37.978348 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:37.978301    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb" podStartSLOduration=1.01044612 podStartE2EDuration="1.978282284s" totalImagesPullingTime="967.836164ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:43:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:43:37.977019058 +0000 UTC m=+825.678725721" watchObservedRunningTime="2026-08-03 23:43:37.978282284 +0000 UTC m=+825.679988948"
Aug 03 23:43:41.733473 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.733431    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"]
Aug 03 23:43:41.738142 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.738124    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.741196 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.741172    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Aug 03 23:43:41.748437 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.748407    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"]
Aug 03 23:43:41.835046 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.835016    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs97w\" (UniqueName: \"kubernetes.io/projected/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-kube-api-access-gs97w\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.835208 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.835077    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.835208 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.835131    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.835208 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.835162    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.835208 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.835200    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.835355 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.835223    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.936652 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.936622    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.936796 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.936662    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.936796 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.936691    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.936796 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.936753    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.936796 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.936789    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.936936 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.936860    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs97w\" (UniqueName: \"kubernetes.io/projected/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-kube-api-access-gs97w\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.937175 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.937150    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.937226 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.937158    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.937226 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.937209    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.939204 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.939183    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.939425 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.939409    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:41.945365 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:41.945345    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs97w\" (UniqueName: \"kubernetes.io/projected/2f818d1b-10d2-40a5-9d66-4fe74d7b133f-kube-api-access-gs97w\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hkqlp\" (UID: \"2f818d1b-10d2-40a5-9d66-4fe74d7b133f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:42.049079 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:42.049017    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:42.173618 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:42.173589    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"]
Aug 03 23:43:42.176863 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:43:42.176822    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f818d1b_10d2_40a5_9d66_4fe74d7b133f.slice/crio-26039e308c0363d2d43fe78d16178064aea28a7b310197c87df818e8adc934df WatchSource:0}: Error finding container 26039e308c0363d2d43fe78d16178064aea28a7b310197c87df818e8adc934df: Status 404 returned error can't find the container with id 26039e308c0363d2d43fe78d16178064aea28a7b310197c87df818e8adc934df
Aug 03 23:43:42.985412 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:42.985380    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp" event={"ID":"2f818d1b-10d2-40a5-9d66-4fe74d7b133f","Type":"ContainerStarted","Data":"5ce07f935ad61add72ac47d3b7e22e161a6f4e46ea796923b2996f498bbaf5d8"}
Aug 03 23:43:42.985794 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:42.985425    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp" event={"ID":"2f818d1b-10d2-40a5-9d66-4fe74d7b133f","Type":"ContainerStarted","Data":"26039e308c0363d2d43fe78d16178064aea28a7b310197c87df818e8adc934df"}
Aug 03 23:43:43.008729 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:43.004689    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp" podStartSLOduration=1.840614724 podStartE2EDuration="2.00466794s" totalImagesPullingTime="164.053216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:43:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:43:43.001414656 +0000 UTC m=+830.703121320" watchObservedRunningTime="2026-08-03 23:43:43.00466794 +0000 UTC m=+830.706374603"
Aug 03 23:43:46.758403 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:46.758375    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:46.764757 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:46.764731    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:46.770744 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:46.770727    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:47.013911 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:47.013833    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29lnb"
Aug 03 23:43:52.049177 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:52.049149    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:52.049618 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:52.049238    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:52.061222 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:52.061203    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:53.041581 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:53.041556    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hkqlp"
Aug 03 23:43:55.155724 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.155683    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"]
Aug 03 23:43:55.158051 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.158030    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.160342 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.160320    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Aug 03 23:43:55.169922 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.169901    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"]
Aug 03 23:43:55.247481 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.247459    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.247592 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.247490    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.247592 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.247517    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bacf1a4-32eb-4388-9ffa-25d0469275dd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.247592 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.247567    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcxkz\" (UniqueName: \"kubernetes.io/projected/6bacf1a4-32eb-4388-9ffa-25d0469275dd-kube-api-access-zcxkz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.247729 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.247645    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.247729 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.247676    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.348800 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.348775    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.348914 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.348811    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.348914 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.348834    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.348914 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.348854    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.348914 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.348880    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bacf1a4-32eb-4388-9ffa-25d0469275dd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.348914 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.348902    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcxkz\" (UniqueName: \"kubernetes.io/projected/6bacf1a4-32eb-4388-9ffa-25d0469275dd-kube-api-access-zcxkz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.349236 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.349219    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.349291 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.349245    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.349291 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.349280    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.351177 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.351155    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bacf1a4-32eb-4388-9ffa-25d0469275dd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.351432 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.351412    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bacf1a4-32eb-4388-9ffa-25d0469275dd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.356487 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.356463    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcxkz\" (UniqueName: \"kubernetes.io/projected/6bacf1a4-32eb-4388-9ffa-25d0469275dd-kube-api-access-zcxkz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pknlc\" (UID: \"6bacf1a4-32eb-4388-9ffa-25d0469275dd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.468032 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.468007    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:43:55.591203 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:55.591053    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"]
Aug 03 23:43:55.593420 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:43:55.593395    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bacf1a4_32eb_4388_9ffa_25d0469275dd.slice/crio-dc630256ddfd45e11aefcb8c5421a13e417a6ccbc78325c987ed3e7a39a48c1d WatchSource:0}: Error finding container dc630256ddfd45e11aefcb8c5421a13e417a6ccbc78325c987ed3e7a39a48c1d: Status 404 returned error can't find the container with id dc630256ddfd45e11aefcb8c5421a13e417a6ccbc78325c987ed3e7a39a48c1d
Aug 03 23:43:56.046166 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:56.046083    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc" event={"ID":"6bacf1a4-32eb-4388-9ffa-25d0469275dd","Type":"ContainerStarted","Data":"b236fce23eaf4ef26871ce9128641a4ac7b5e9ead532b2838ce8e1bb7fd8d22c"}
Aug 03 23:43:56.046166 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:56.046142    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc" event={"ID":"6bacf1a4-32eb-4388-9ffa-25d0469275dd","Type":"ContainerStarted","Data":"dc630256ddfd45e11aefcb8c5421a13e417a6ccbc78325c987ed3e7a39a48c1d"}
Aug 03 23:43:56.063290 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:43:56.063249    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc" podStartSLOduration=0.869023721 podStartE2EDuration="1.063232496s" totalImagesPullingTime="194.208775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:43:56.061653274 +0000 UTC m=+843.763359938" watchObservedRunningTime="2026-08-03 23:43:56.063232496 +0000 UTC m=+843.764939159"
Aug 03 23:44:05.469091 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:44:05.469054    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:44:05.469463 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:44:05.469110    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:44:05.481027 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:44:05.480996    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:44:06.102884 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:44:06.102856    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pknlc"
Aug 03 23:46:12.964802 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:12.964770    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-7776h"]
Aug 03 23:46:12.965225 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:12.964981    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-96bff6bb9-7776h" podUID="74fa132a-b2b2-4781-aefd-ecdf7c89d87c" containerName="manager" containerID="cri-o://732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f" gracePeriod=30
Aug 03 23:46:13.212975 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.212953    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:46:13.316747 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.316640    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7fl2\" (UniqueName: \"kubernetes.io/projected/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-kube-api-access-h7fl2\") pod \"74fa132a-b2b2-4781-aefd-ecdf7c89d87c\" (UID: \"74fa132a-b2b2-4781-aefd-ecdf7c89d87c\") "
Aug 03 23:46:13.316747 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.316685    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-webhook-cert\") pod \"74fa132a-b2b2-4781-aefd-ecdf7c89d87c\" (UID: \"74fa132a-b2b2-4781-aefd-ecdf7c89d87c\") "
Aug 03 23:46:13.318773 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.318745    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "74fa132a-b2b2-4781-aefd-ecdf7c89d87c" (UID: "74fa132a-b2b2-4781-aefd-ecdf7c89d87c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:46:13.318876 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.318770    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-kube-api-access-h7fl2" (OuterVolumeSpecName: "kube-api-access-h7fl2") pod "74fa132a-b2b2-4781-aefd-ecdf7c89d87c" (UID: "74fa132a-b2b2-4781-aefd-ecdf7c89d87c"). InnerVolumeSpecName "kube-api-access-h7fl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:46:13.417983 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.417960    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7fl2\" (UniqueName: \"kubernetes.io/projected/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-kube-api-access-h7fl2\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:46:13.417983 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.417981    2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74fa132a-b2b2-4781-aefd-ecdf7c89d87c-webhook-cert\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:46:13.675692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.675606    2553 generic.go:356] "Generic (PLEG): container finished" podID="74fa132a-b2b2-4781-aefd-ecdf7c89d87c" containerID="732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f" exitCode=0
Aug 03 23:46:13.675692 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.675650    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-96bff6bb9-7776h"
Aug 03 23:46:13.675840 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.675691    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-96bff6bb9-7776h" event={"ID":"74fa132a-b2b2-4781-aefd-ecdf7c89d87c","Type":"ContainerDied","Data":"732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f"}
Aug 03 23:46:13.675840 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.675755    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-96bff6bb9-7776h" event={"ID":"74fa132a-b2b2-4781-aefd-ecdf7c89d87c","Type":"ContainerDied","Data":"cd2d993e90ebca6735014e891b1d3448308c82dca2db8dc4d66fab39f8cd2a6f"}
Aug 03 23:46:13.675840 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.675778    2553 scope.go:117] "RemoveContainer" containerID="732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f"
Aug 03 23:46:13.684903 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.684888    2553 scope.go:117] "RemoveContainer" containerID="732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f"
Aug 03 23:46:13.685164 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:46:13.685146    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f\": container with ID starting with 732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f not found: ID does not exist" containerID="732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f"
Aug 03 23:46:13.685226 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.685173    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f"} err="failed to get container status \"732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f\": rpc error: code = NotFound desc = could not find container \"732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f\": container with ID starting with 732369a3abf95d712656840745917ded84730181cb0733d7a892ef91ba5b781f not found: ID does not exist"
Aug 03 23:46:13.697050 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.697029    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-7776h"]
Aug 03 23:46:13.698783 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:13.698764    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-7776h"]
Aug 03 23:46:14.763095 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:46:14.763065    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74fa132a-b2b2-4781-aefd-ecdf7c89d87c" path="/var/lib/kubelet/pods/74fa132a-b2b2-4781-aefd-ecdf7c89d87c/volumes"
Aug 03 23:47:14.465946 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.465914    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-96bff6bb9-pvqd6"]
Aug 03 23:47:14.466339 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.466287    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74fa132a-b2b2-4781-aefd-ecdf7c89d87c" containerName="manager"
Aug 03 23:47:14.466339 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.466298    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="74fa132a-b2b2-4781-aefd-ecdf7c89d87c" containerName="manager"
Aug 03 23:47:14.466413 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.466367    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="74fa132a-b2b2-4781-aefd-ecdf7c89d87c" containerName="manager"
Aug 03 23:47:14.468838 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.468822    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:14.471154 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.471136    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wwmmf\""
Aug 03 23:47:14.472150 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.472136    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 03 23:47:14.472252 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.472237    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 03 23:47:14.476795 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.476773    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-pvqd6"]
Aug 03 23:47:14.617468 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.617437    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2qkd\" (UniqueName: \"kubernetes.io/projected/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-kube-api-access-p2qkd\") pod \"maas-controller-96bff6bb9-pvqd6\" (UID: \"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8\") " pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:14.617627 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.617488    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-webhook-cert\") pod \"maas-controller-96bff6bb9-pvqd6\" (UID: \"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8\") " pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:14.718885 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.718810    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-webhook-cert\") pod \"maas-controller-96bff6bb9-pvqd6\" (UID: \"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8\") " pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:14.719013 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.718917    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2qkd\" (UniqueName: \"kubernetes.io/projected/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-kube-api-access-p2qkd\") pod \"maas-controller-96bff6bb9-pvqd6\" (UID: \"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8\") " pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:14.721361 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.721336    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-webhook-cert\") pod \"maas-controller-96bff6bb9-pvqd6\" (UID: \"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8\") " pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:14.728212 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.728186    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2qkd\" (UniqueName: \"kubernetes.io/projected/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-kube-api-access-p2qkd\") pod \"maas-controller-96bff6bb9-pvqd6\" (UID: \"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8\") " pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:14.779140 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.779118    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:14.906809 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.906776    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-pvqd6"]
Aug 03 23:47:14.909043 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:47:14.909011    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c3b6c6_0fe9_4eaa_8422_a042efc7c9d8.slice/crio-d8dbc6913be64895c22bea540e69145bb6b1ef256dff1748c55b980726d70b63 WatchSource:0}: Error finding container d8dbc6913be64895c22bea540e69145bb6b1ef256dff1748c55b980726d70b63: Status 404 returned error can't find the container with id d8dbc6913be64895c22bea540e69145bb6b1ef256dff1748c55b980726d70b63
Aug 03 23:47:14.910371 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.910356    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 23:47:14.954362 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:14.954333    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-96bff6bb9-pvqd6" event={"ID":"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8","Type":"ContainerStarted","Data":"d8dbc6913be64895c22bea540e69145bb6b1ef256dff1748c55b980726d70b63"}
Aug 03 23:47:15.960305 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:15.960269    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-96bff6bb9-pvqd6" event={"ID":"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8","Type":"ContainerStarted","Data":"aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a"}
Aug 03 23:47:15.960660 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:15.960359    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:15.977739 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:15.977683    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-96bff6bb9-pvqd6" podStartSLOduration=1.6114106179999999 podStartE2EDuration="1.977669243s" totalImagesPullingTime="366.258625ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:47:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:47:15.976676468 +0000 UTC m=+1043.678383132" watchObservedRunningTime="2026-08-03 23:47:15.977669243 +0000 UTC m=+1043.679375906"
Aug 03 23:47:26.966246 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:26.966218    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:31.607372 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:31.607336    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-pvqd6"]
Aug 03 23:47:31.607775 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:31.607557    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-96bff6bb9-pvqd6" podUID="b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8" containerName="manager" containerID="cri-o://aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a" gracePeriod=30
Aug 03 23:47:31.854332 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:31.854307    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:31.968605 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:31.968580    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2qkd\" (UniqueName: \"kubernetes.io/projected/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-kube-api-access-p2qkd\") pod \"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8\" (UID: \"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8\") "
Aug 03 23:47:31.968774 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:31.968640    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-webhook-cert\") pod \"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8\" (UID: \"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8\") "
Aug 03 23:47:31.970670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:31.970645    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8" (UID: "b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 23:47:31.970670 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:31.970653    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-kube-api-access-p2qkd" (OuterVolumeSpecName: "kube-api-access-p2qkd") pod "b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8" (UID: "b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8"). InnerVolumeSpecName "kube-api-access-p2qkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 23:47:32.032501 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.032475    2553 generic.go:356] "Generic (PLEG): container finished" podID="b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8" containerID="aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a" exitCode=0
Aug 03 23:47:32.032609 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.032515    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-96bff6bb9-pvqd6"
Aug 03 23:47:32.032609 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.032551    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-96bff6bb9-pvqd6" event={"ID":"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8","Type":"ContainerDied","Data":"aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a"}
Aug 03 23:47:32.032609 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.032600    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-96bff6bb9-pvqd6" event={"ID":"b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8","Type":"ContainerDied","Data":"d8dbc6913be64895c22bea540e69145bb6b1ef256dff1748c55b980726d70b63"}
Aug 03 23:47:32.032760 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.032621    2553 scope.go:117] "RemoveContainer" containerID="aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a"
Aug 03 23:47:32.043574 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.043557    2553 scope.go:117] "RemoveContainer" containerID="aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a"
Aug 03 23:47:32.044039 ip-10-0-143-183 kubenswrapper[2553]: E0803 23:47:32.044019    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a\": container with ID starting with aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a not found: ID does not exist" containerID="aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a"
Aug 03 23:47:32.044110 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.044049    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a"} err="failed to get container status \"aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a\": rpc error: code = NotFound desc = could not find container \"aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a\": container with ID starting with aa12bb8144a642cfff549c1e76dbea417f2097b11f0336fcae58a5d28fae0c8a not found: ID does not exist"
Aug 03 23:47:32.053364 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.053340    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-pvqd6"]
Aug 03 23:47:32.055654 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.055631    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-pvqd6"]
Aug 03 23:47:32.069202 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.069182    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2qkd\" (UniqueName: \"kubernetes.io/projected/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-kube-api-access-p2qkd\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:47:32.069283 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.069204    2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8-webhook-cert\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 03 23:47:32.763893 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.763865    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8" path="/var/lib/kubelet/pods/b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8/volumes"
Aug 03 23:47:32.865198 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.865174    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-96bff6bb9-hz6tn"]
Aug 03 23:47:32.865527 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.865514    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8" containerName="manager"
Aug 03 23:47:32.865570 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.865530    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8" containerName="manager"
Aug 03 23:47:32.865616 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.865597    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5c3b6c6-0fe9-4eaa-8422-a042efc7c9d8" containerName="manager"
Aug 03 23:47:32.867674 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.867656    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-96bff6bb9-hz6tn"
Aug 03 23:47:32.870076 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.870053    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 03 23:47:32.870194 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.870121    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wwmmf\""
Aug 03 23:47:32.871208 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.871194    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 03 23:47:32.892388 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.892364    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-hz6tn"]
Aug 03 23:47:32.976636 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.976606    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf5z9\" (UniqueName: \"kubernetes.io/projected/570ee62e-722b-4af2-a1a8-f4093fe0e5bf-kube-api-access-wf5z9\") pod \"maas-controller-96bff6bb9-hz6tn\" (UID: \"570ee62e-722b-4af2-a1a8-f4093fe0e5bf\") " pod="opendatahub/maas-controller-96bff6bb9-hz6tn"
Aug 03 23:47:32.976824 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:32.976673    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/570ee62e-722b-4af2-a1a8-f4093fe0e5bf-webhook-cert\") pod \"maas-controller-96bff6bb9-hz6tn\" (UID: \"570ee62e-722b-4af2-a1a8-f4093fe0e5bf\") " pod="opendatahub/maas-controller-96bff6bb9-hz6tn"
Aug 03 23:47:33.077884 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:33.077809    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf5z9\" (UniqueName: \"kubernetes.io/projected/570ee62e-722b-4af2-a1a8-f4093fe0e5bf-kube-api-access-wf5z9\") pod \"maas-controller-96bff6bb9-hz6tn\" (UID: \"570ee62e-722b-4af2-a1a8-f4093fe0e5bf\") " pod="opendatahub/maas-controller-96bff6bb9-hz6tn"
Aug 03 23:47:33.077884 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:33.077867    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/570ee62e-722b-4af2-a1a8-f4093fe0e5bf-webhook-cert\") pod \"maas-controller-96bff6bb9-hz6tn\" (UID: \"570ee62e-722b-4af2-a1a8-f4093fe0e5bf\") " pod="opendatahub/maas-controller-96bff6bb9-hz6tn"
Aug 03 23:47:33.080348 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:33.080331    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/570ee62e-722b-4af2-a1a8-f4093fe0e5bf-webhook-cert\") pod \"maas-controller-96bff6bb9-hz6tn\" (UID: \"570ee62e-722b-4af2-a1a8-f4093fe0e5bf\") " pod="opendatahub/maas-controller-96bff6bb9-hz6tn"
Aug 03 23:47:33.086352 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:33.086328    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf5z9\" (UniqueName: \"kubernetes.io/projected/570ee62e-722b-4af2-a1a8-f4093fe0e5bf-kube-api-access-wf5z9\") pod \"maas-controller-96bff6bb9-hz6tn\" (UID: \"570ee62e-722b-4af2-a1a8-f4093fe0e5bf\") " pod="opendatahub/maas-controller-96bff6bb9-hz6tn"
Aug 03 23:47:33.176809 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:33.176786    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-96bff6bb9-hz6tn"
Aug 03 23:47:33.305995 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:33.305972    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-96bff6bb9-hz6tn"]
Aug 03 23:47:33.308023 ip-10-0-143-183 kubenswrapper[2553]: W0803 23:47:33.307991    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570ee62e_722b_4af2_a1a8_f4093fe0e5bf.slice/crio-814adc5513eb2ef9fe602dd794fd7b2ef7804c50f739a814c898c9a328437065 WatchSource:0}: Error finding container 814adc5513eb2ef9fe602dd794fd7b2ef7804c50f739a814c898c9a328437065: Status 404 returned error can't find the container with id 814adc5513eb2ef9fe602dd794fd7b2ef7804c50f739a814c898c9a328437065
Aug 03 23:47:34.044242 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:34.044205    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-96bff6bb9-hz6tn" event={"ID":"570ee62e-722b-4af2-a1a8-f4093fe0e5bf","Type":"ContainerStarted","Data":"601d0acabb7509b54c022f6949141e2dd01ae63891dadcd136c322f44b01f5a7"}
Aug 03 23:47:34.044698 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:34.044274    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-96bff6bb9-hz6tn" event={"ID":"570ee62e-722b-4af2-a1a8-f4093fe0e5bf","Type":"ContainerStarted","Data":"814adc5513eb2ef9fe602dd794fd7b2ef7804c50f739a814c898c9a328437065"}
Aug 03 23:47:34.044698 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:34.044301    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-96bff6bb9-hz6tn"
Aug 03 23:47:34.061388 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:34.061345    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-96bff6bb9-hz6tn" podStartSLOduration=1.6996267 podStartE2EDuration="2.0613304s" totalImagesPullingTime="361.7037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 23:47:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 23:47:34.058964408 +0000 UTC m=+1061.760671071" watchObservedRunningTime="2026-08-03 23:47:34.0613304 +0000 UTC m=+1061.763037062"
Aug 03 23:47:45.050546 ip-10-0-143-183 kubenswrapper[2553]: I0803 23:47:45.050466    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-96bff6bb9-hz6tn"
Aug 04 00:00:00.144396 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.144365    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29763360-5x87r"]
Aug 04 00:00:00.147889 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.147874    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29763360-5x87r"
Aug 04 00:00:00.150240 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.150216    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\""
Aug 04 00:00:00.150372 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.150352    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-jq7nw\""
Aug 04 00:00:00.155682 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.155656    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29763360-5x87r"]
Aug 04 00:00:00.289922 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.289890    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6414e19d-e976-458a-ade8-85a8f9bb2443-serviceca\") pod \"image-pruner-29763360-5x87r\" (UID: \"6414e19d-e976-458a-ade8-85a8f9bb2443\") " pod="openshift-image-registry/image-pruner-29763360-5x87r"
Aug 04 00:00:00.289922 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.289924    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtglh\" (UniqueName: \"kubernetes.io/projected/6414e19d-e976-458a-ade8-85a8f9bb2443-kube-api-access-rtglh\") pod \"image-pruner-29763360-5x87r\" (UID: \"6414e19d-e976-458a-ade8-85a8f9bb2443\") " pod="openshift-image-registry/image-pruner-29763360-5x87r"
Aug 04 00:00:00.391186 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.391157    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6414e19d-e976-458a-ade8-85a8f9bb2443-serviceca\") pod \"image-pruner-29763360-5x87r\" (UID: \"6414e19d-e976-458a-ade8-85a8f9bb2443\") " pod="openshift-image-registry/image-pruner-29763360-5x87r"
Aug 04 00:00:00.391364 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.391191    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtglh\" (UniqueName: \"kubernetes.io/projected/6414e19d-e976-458a-ade8-85a8f9bb2443-kube-api-access-rtglh\") pod \"image-pruner-29763360-5x87r\" (UID: \"6414e19d-e976-458a-ade8-85a8f9bb2443\") " pod="openshift-image-registry/image-pruner-29763360-5x87r"
Aug 04 00:00:00.391762 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.391741    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6414e19d-e976-458a-ade8-85a8f9bb2443-serviceca\") pod \"image-pruner-29763360-5x87r\" (UID: \"6414e19d-e976-458a-ade8-85a8f9bb2443\") " pod="openshift-image-registry/image-pruner-29763360-5x87r"
Aug 04 00:00:00.399872 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.399811    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtglh\" (UniqueName: \"kubernetes.io/projected/6414e19d-e976-458a-ade8-85a8f9bb2443-kube-api-access-rtglh\") pod \"image-pruner-29763360-5x87r\" (UID: \"6414e19d-e976-458a-ade8-85a8f9bb2443\") " pod="openshift-image-registry/image-pruner-29763360-5x87r"
Aug 04 00:00:00.476622 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.476587    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29763360-5x87r"
Aug 04 00:00:00.609220 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:00.609196    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29763360-5x87r"]
Aug 04 00:00:00.611295 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:00:00.611266    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6414e19d_e976_458a_ade8_85a8f9bb2443.slice/crio-2149de756e001a32e441e703ddb92016e1c04f95fd8330a59f1415a354aebaac WatchSource:0}: Error finding container 2149de756e001a32e441e703ddb92016e1c04f95fd8330a59f1415a354aebaac: Status 404 returned error can't find the container with id 2149de756e001a32e441e703ddb92016e1c04f95fd8330a59f1415a354aebaac
Aug 04 00:00:01.497149 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:01.497116    2553 generic.go:356] "Generic (PLEG): container finished" podID="6414e19d-e976-458a-ade8-85a8f9bb2443" containerID="0fbadcc13fe3c2d60085f0cf3c9116db185b93a6262a8c3461e117c85e4cd0df" exitCode=0
Aug 04 00:00:01.497508 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:01.497217    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29763360-5x87r" event={"ID":"6414e19d-e976-458a-ade8-85a8f9bb2443","Type":"ContainerDied","Data":"0fbadcc13fe3c2d60085f0cf3c9116db185b93a6262a8c3461e117c85e4cd0df"}
Aug 04 00:00:01.497508 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:01.497265    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29763360-5x87r" event={"ID":"6414e19d-e976-458a-ade8-85a8f9bb2443","Type":"ContainerStarted","Data":"2149de756e001a32e441e703ddb92016e1c04f95fd8330a59f1415a354aebaac"}
Aug 04 00:00:02.635400 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:02.635378    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29763360-5x87r"
Aug 04 00:00:02.712605 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:02.712577    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6414e19d-e976-458a-ade8-85a8f9bb2443-serviceca\") pod \"6414e19d-e976-458a-ade8-85a8f9bb2443\" (UID: \"6414e19d-e976-458a-ade8-85a8f9bb2443\") "
Aug 04 00:00:02.712752 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:02.712686    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtglh\" (UniqueName: \"kubernetes.io/projected/6414e19d-e976-458a-ade8-85a8f9bb2443-kube-api-access-rtglh\") pod \"6414e19d-e976-458a-ade8-85a8f9bb2443\" (UID: \"6414e19d-e976-458a-ade8-85a8f9bb2443\") "
Aug 04 00:00:02.713220 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:02.713195    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6414e19d-e976-458a-ade8-85a8f9bb2443-serviceca" (OuterVolumeSpecName: "serviceca") pod "6414e19d-e976-458a-ade8-85a8f9bb2443" (UID: "6414e19d-e976-458a-ade8-85a8f9bb2443"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:00:02.714958 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:02.714934    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6414e19d-e976-458a-ade8-85a8f9bb2443-kube-api-access-rtglh" (OuterVolumeSpecName: "kube-api-access-rtglh") pod "6414e19d-e976-458a-ade8-85a8f9bb2443" (UID: "6414e19d-e976-458a-ade8-85a8f9bb2443"). InnerVolumeSpecName "kube-api-access-rtglh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:00:02.813317 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:02.813293    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtglh\" (UniqueName: \"kubernetes.io/projected/6414e19d-e976-458a-ade8-85a8f9bb2443-kube-api-access-rtglh\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:00:02.813317 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:02.813316    2553 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6414e19d-e976-458a-ade8-85a8f9bb2443-serviceca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:00:03.507700 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:03.507671    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29763360-5x87r"
Aug 04 00:00:03.507888 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:03.507700    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29763360-5x87r" event={"ID":"6414e19d-e976-458a-ade8-85a8f9bb2443","Type":"ContainerDied","Data":"2149de756e001a32e441e703ddb92016e1c04f95fd8330a59f1415a354aebaac"}
Aug 04 00:00:03.507888 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:00:03.507763    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2149de756e001a32e441e703ddb92016e1c04f95fd8330a59f1415a354aebaac"
Aug 04 00:02:59.718607 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.718530    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"]
Aug 04 00:02:59.719074 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.718910    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6414e19d-e976-458a-ade8-85a8f9bb2443" containerName="image-pruner"
Aug 04 00:02:59.719074 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.718920    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="6414e19d-e976-458a-ade8-85a8f9bb2443" containerName="image-pruner"
Aug 04 00:02:59.719074 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.718985    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="6414e19d-e976-458a-ade8-85a8f9bb2443" containerName="image-pruner"
Aug 04 00:02:59.722071 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.722050    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:02:59.724834 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.724815    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f9c1edcb-serving-cert\""
Aug 04 00:02:59.736747 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.736693    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"]
Aug 04 00:02:59.777699 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.777675    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-maas-api-tls\") pod \"maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:02:59.777866 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.777767    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-openshift-service-ca\") pod \"maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:02:59.777866 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.777799    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfnb5\" (UniqueName: \"kubernetes.io/projected/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-kube-api-access-bfnb5\") pod \"maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:02:59.879270 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.879238    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-maas-api-tls\") pod \"maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:02:59.879444 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.879283    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-openshift-service-ca\") pod \"maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:02:59.879444 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.879308    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnb5\" (UniqueName: \"kubernetes.io/projected/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-kube-api-access-bfnb5\") pod \"maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:02:59.879444 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:02:59.879404    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-serving-cert: secret "maas-api-e2e-ait-f9c1edcb-serving-cert" not found
Aug 04 00:02:59.879612 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:02:59.879487    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-maas-api-tls podName:429dfc4f-41ba-42c4-b59b-dcdc9a9a7395 nodeName:}" failed. No retries permitted until 2026-08-04 00:03:00.379467002 +0000 UTC m=+1988.081173643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-maas-api-tls") pod "maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd" (UID: "429dfc4f-41ba-42c4-b59b-dcdc9a9a7395") : secret "maas-api-e2e-ait-f9c1edcb-serving-cert" not found
Aug 04 00:02:59.879898 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.879879    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-openshift-service-ca\") pod \"maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:02:59.887701 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:02:59.887678    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfnb5\" (UniqueName: \"kubernetes.io/projected/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-kube-api-access-bfnb5\") pod \"maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:03:00.384592 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.384556    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-maas-api-tls\") pod \"maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:03:00.386965 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.386946    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-maas-api-tls\") pod \"maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:03:00.440603 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.440564    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"]
Aug 04 00:03:00.444201 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.444179    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:00.446460 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.446439    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f9c1edcb\""
Aug 04 00:03:00.446560 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.446439    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f9c1edcb-dockercfg-ftxpz\""
Aug 04 00:03:00.454896 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.454874    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"]
Aug 04 00:03:00.475916 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.475889    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"]
Aug 04 00:03:00.479269 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.479252    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:00.488778 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.488755    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"]
Aug 04 00:03:00.586059 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.586028    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2d7w\" (UniqueName: \"kubernetes.io/projected/28b51def-e1d3-49ae-b793-dcdaba596e68-kube-api-access-w2d7w\") pod \"payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk\" (UID: \"28b51def-e1d3-49ae-b793-dcdaba596e68\") " pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:00.586260 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.586066    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b51def-e1d3-49ae-b793-dcdaba596e68-plugins-config-volume\") pod \"payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk\" (UID: \"28b51def-e1d3-49ae-b793-dcdaba596e68\") " pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:00.586260 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.586085    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4\" (UID: \"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:00.586260 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.586123    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-749nr\" (UniqueName: \"kubernetes.io/projected/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-kube-api-access-749nr\") pod \"payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4\" (UID: \"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:00.631545 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.631514    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:03:00.687129 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.687082    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2d7w\" (UniqueName: \"kubernetes.io/projected/28b51def-e1d3-49ae-b793-dcdaba596e68-kube-api-access-w2d7w\") pod \"payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk\" (UID: \"28b51def-e1d3-49ae-b793-dcdaba596e68\") " pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:00.687347 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.687320    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b51def-e1d3-49ae-b793-dcdaba596e68-plugins-config-volume\") pod \"payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk\" (UID: \"28b51def-e1d3-49ae-b793-dcdaba596e68\") " pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:00.687488 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.687460    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4\" (UID: \"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:00.687677 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.687648    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-749nr\" (UniqueName: \"kubernetes.io/projected/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-kube-api-access-749nr\") pod \"payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4\" (UID: \"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:00.688345 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.688315    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b51def-e1d3-49ae-b793-dcdaba596e68-plugins-config-volume\") pod \"payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk\" (UID: \"28b51def-e1d3-49ae-b793-dcdaba596e68\") " pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:00.689577 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.689558    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4\" (UID: \"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:00.698070 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.695988    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2d7w\" (UniqueName: \"kubernetes.io/projected/28b51def-e1d3-49ae-b793-dcdaba596e68-kube-api-access-w2d7w\") pod \"payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk\" (UID: \"28b51def-e1d3-49ae-b793-dcdaba596e68\") " pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:00.698070 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.698045    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-749nr\" (UniqueName: \"kubernetes.io/projected/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-kube-api-access-749nr\") pod \"payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4\" (UID: \"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:00.756230 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.756196    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:00.790992 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.790964    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:00.901079 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.901007    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"]
Aug 04 00:03:00.904574 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:03:00.904537    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87b92da_6c3c_4e8d_93ec_f0ad65ff6037.slice/crio-dbac49216f952200556660e51bd6fd830c56e6d9fd2a00be98a5e8377cd47407 WatchSource:0}: Error finding container dbac49216f952200556660e51bd6fd830c56e6d9fd2a00be98a5e8377cd47407: Status 404 returned error can't find the container with id dbac49216f952200556660e51bd6fd830c56e6d9fd2a00be98a5e8377cd47407
Aug 04 00:03:00.938048 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.938020    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"]
Aug 04 00:03:00.974859 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.974829    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"]
Aug 04 00:03:00.975998 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:03:00.975976    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod429dfc4f_41ba_42c4_b59b_dcdc9a9a7395.slice/crio-10bf1f74e2f7ab15e7710b0432950145a7443fbd81588b99a977e7ba077d3479 WatchSource:0}: Error finding container 10bf1f74e2f7ab15e7710b0432950145a7443fbd81588b99a977e7ba077d3479: Status 404 returned error can't find the container with id 10bf1f74e2f7ab15e7710b0432950145a7443fbd81588b99a977e7ba077d3479
Aug 04 00:03:00.977611 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:00.977578    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 00:03:01.343556 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:01.343519    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4" event={"ID":"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037","Type":"ContainerStarted","Data":"cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8"}
Aug 04 00:03:01.343776 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:01.343591    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:01.343776 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:01.343615    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4" event={"ID":"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037","Type":"ContainerStarted","Data":"dbac49216f952200556660e51bd6fd830c56e6d9fd2a00be98a5e8377cd47407"}
Aug 04 00:03:01.344789 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:01.344761    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd" event={"ID":"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395","Type":"ContainerStarted","Data":"10bf1f74e2f7ab15e7710b0432950145a7443fbd81588b99a977e7ba077d3479"}
Aug 04 00:03:01.346185 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:01.346165    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk" event={"ID":"28b51def-e1d3-49ae-b793-dcdaba596e68","Type":"ContainerStarted","Data":"34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493"}
Aug 04 00:03:01.346297 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:01.346197    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk" event={"ID":"28b51def-e1d3-49ae-b793-dcdaba596e68","Type":"ContainerStarted","Data":"5fdadf2db733ea0916b72dacfe7b60fc568ddb85bad88a2ee86c222d003f7e34"}
Aug 04 00:03:01.346297 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:01.346226    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:01.359088 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:01.359048    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4" podStartSLOduration=1.359036114 podStartE2EDuration="1.359036114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:03:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:01.35887783 +0000 UTC m=+1989.060584493" watchObservedRunningTime="2026-08-04 00:03:01.359036114 +0000 UTC m=+1989.060742776"
Aug 04 00:03:01.376593 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:01.376552    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk" podStartSLOduration=1.376538627 podStartE2EDuration="1.376538627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:03:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:01.376071131 +0000 UTC m=+1989.077777796" watchObservedRunningTime="2026-08-04 00:03:01.376538627 +0000 UTC m=+1989.078245290"
Aug 04 00:03:02.352361 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:02.352329    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd" event={"ID":"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395","Type":"ContainerStarted","Data":"41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f"}
Aug 04 00:03:02.370725 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:02.370652    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd" podStartSLOduration=2.922833943 podStartE2EDuration="3.370637356s" totalImagesPullingTime="447.803413ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:02.367821951 +0000 UTC m=+1990.069528616" watchObservedRunningTime="2026-08-04 00:03:02.370637356 +0000 UTC m=+1990.072344017"
Aug 04 00:03:03.357317 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:03.357283    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:03:09.366938 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:09.366912    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:03:12.354369 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:12.354341    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:12.354835 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:12.354424    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:40.192270 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.192241    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"]
Aug 04 00:03:40.192935 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.192753    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd" podUID="429dfc4f-41ba-42c4-b59b-dcdc9a9a7395" containerName="maas-api" containerID="cri-o://41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f" gracePeriod=30
Aug 04 00:03:40.236658 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.236629    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"]
Aug 04 00:03:40.237120 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.237060    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk" podUID="28b51def-e1d3-49ae-b793-dcdaba596e68" containerName="payload-processing" containerID="cri-o://34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493" gracePeriod=30
Aug 04 00:03:40.240651 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.240606    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"]
Aug 04 00:03:40.240952 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.240927    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4" podUID="b87b92da-6c3c-4e8d-93ec-f0ad65ff6037" containerName="payload-pre-processing" containerID="cri-o://cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8" gracePeriod=30
Aug 04 00:03:40.485789 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.485763    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:03:40.529346 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.529316    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:40.532800 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.532777    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:40.539085 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.539061    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfnb5\" (UniqueName: \"kubernetes.io/projected/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-kube-api-access-bfnb5\") pod \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") "
Aug 04 00:03:40.539192 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.539147    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-maas-api-tls\") pod \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") "
Aug 04 00:03:40.539254 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.539220    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-openshift-service-ca\") pod \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\" (UID: \"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395\") "
Aug 04 00:03:40.539581 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.539556    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "429dfc4f-41ba-42c4-b59b-dcdc9a9a7395" (UID: "429dfc4f-41ba-42c4-b59b-dcdc9a9a7395"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:03:40.541764 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.541726    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-kube-api-access-bfnb5" (OuterVolumeSpecName: "kube-api-access-bfnb5") pod "429dfc4f-41ba-42c4-b59b-dcdc9a9a7395" (UID: "429dfc4f-41ba-42c4-b59b-dcdc9a9a7395"). InnerVolumeSpecName "kube-api-access-bfnb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:03:40.541853 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.541787    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "429dfc4f-41ba-42c4-b59b-dcdc9a9a7395" (UID: "429dfc4f-41ba-42c4-b59b-dcdc9a9a7395"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:03:40.542154 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.542134    2553 generic.go:356] "Generic (PLEG): container finished" podID="28b51def-e1d3-49ae-b793-dcdaba596e68" containerID="34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493" exitCode=0
Aug 04 00:03:40.542236 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.542173    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"
Aug 04 00:03:40.542236 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.542212    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk" event={"ID":"28b51def-e1d3-49ae-b793-dcdaba596e68","Type":"ContainerDied","Data":"34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493"}
Aug 04 00:03:40.542333 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.542258    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk" event={"ID":"28b51def-e1d3-49ae-b793-dcdaba596e68","Type":"ContainerDied","Data":"5fdadf2db733ea0916b72dacfe7b60fc568ddb85bad88a2ee86c222d003f7e34"}
Aug 04 00:03:40.542333 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.542289    2553 scope.go:117] "RemoveContainer" containerID="34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493"
Aug 04 00:03:40.543507 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.543491    2553 generic.go:356] "Generic (PLEG): container finished" podID="b87b92da-6c3c-4e8d-93ec-f0ad65ff6037" containerID="cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8" exitCode=0
Aug 04 00:03:40.543608 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.543560    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4" event={"ID":"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037","Type":"ContainerDied","Data":"cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8"}
Aug 04 00:03:40.543687 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.543608    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4" event={"ID":"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037","Type":"ContainerDied","Data":"dbac49216f952200556660e51bd6fd830c56e6d9fd2a00be98a5e8377cd47407"}
Aug 04 00:03:40.543687 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.543615    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"
Aug 04 00:03:40.544928 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.544908    2553 generic.go:356] "Generic (PLEG): container finished" podID="429dfc4f-41ba-42c4-b59b-dcdc9a9a7395" containerID="41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f" exitCode=0
Aug 04 00:03:40.545016 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.544942    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"
Aug 04 00:03:40.545016 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.544944    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd" event={"ID":"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395","Type":"ContainerDied","Data":"41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f"}
Aug 04 00:03:40.545016 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.545000    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd" event={"ID":"429dfc4f-41ba-42c4-b59b-dcdc9a9a7395","Type":"ContainerDied","Data":"10bf1f74e2f7ab15e7710b0432950145a7443fbd81588b99a977e7ba077d3479"}
Aug 04 00:03:40.552204 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.552186    2553 scope.go:117] "RemoveContainer" containerID="34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493"
Aug 04 00:03:40.552472 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:03:40.552449    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493\": container with ID starting with 34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493 not found: ID does not exist" containerID="34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493"
Aug 04 00:03:40.552533 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.552485    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493"} err="failed to get container status \"34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493\": rpc error: code = NotFound desc = could not find container \"34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493\": container with ID starting with 34ac58e2d5131ca4dc0c9f544bac88d07236c645530bf410f29c01d91be95493 not found: ID does not exist"
Aug 04 00:03:40.552533 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.552512    2553 scope.go:117] "RemoveContainer" containerID="cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8"
Aug 04 00:03:40.562293 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.562273    2553 scope.go:117] "RemoveContainer" containerID="cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8"
Aug 04 00:03:40.562616 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:03:40.562591    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8\": container with ID starting with cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8 not found: ID does not exist" containerID="cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8"
Aug 04 00:03:40.562731 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.562623    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8"} err="failed to get container status \"cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8\": rpc error: code = NotFound desc = could not find container \"cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8\": container with ID starting with cf766f89174601ac31dffb8f7eaf0bca85352c0e9035756f4514f7634edfe3c8 not found: ID does not exist"
Aug 04 00:03:40.562731 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.562645    2553 scope.go:117] "RemoveContainer" containerID="41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f"
Aug 04 00:03:40.570330 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.570307    2553 scope.go:117] "RemoveContainer" containerID="41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f"
Aug 04 00:03:40.570569 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:03:40.570551    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f\": container with ID starting with 41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f not found: ID does not exist" containerID="41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f"
Aug 04 00:03:40.570630 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.570576    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f"} err="failed to get container status \"41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f\": rpc error: code = NotFound desc = could not find container \"41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f\": container with ID starting with 41246a84ae8fa6fc1637aa09226bda485f6da09b1863aa8f5f502d457d5b3e6f not found: ID does not exist"
Aug 04 00:03:40.640593 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.640554    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-749nr\" (UniqueName: \"kubernetes.io/projected/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-kube-api-access-749nr\") pod \"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037\" (UID: \"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037\") "
Aug 04 00:03:40.640758 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.640662    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-plugins-config-volume\") pod \"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037\" (UID: \"b87b92da-6c3c-4e8d-93ec-f0ad65ff6037\") "
Aug 04 00:03:40.640758 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.640701    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2d7w\" (UniqueName: \"kubernetes.io/projected/28b51def-e1d3-49ae-b793-dcdaba596e68-kube-api-access-w2d7w\") pod \"28b51def-e1d3-49ae-b793-dcdaba596e68\" (UID: \"28b51def-e1d3-49ae-b793-dcdaba596e68\") "
Aug 04 00:03:40.640758 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.640750    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b51def-e1d3-49ae-b793-dcdaba596e68-plugins-config-volume\") pod \"28b51def-e1d3-49ae-b793-dcdaba596e68\" (UID: \"28b51def-e1d3-49ae-b793-dcdaba596e68\") "
Aug 04 00:03:40.641113 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.641091    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:03:40.641113 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.641114    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfnb5\" (UniqueName: \"kubernetes.io/projected/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-kube-api-access-bfnb5\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:03:40.641281 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.641130    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:03:40.641281 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.641142    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b87b92da-6c3c-4e8d-93ec-f0ad65ff6037" (UID: "b87b92da-6c3c-4e8d-93ec-f0ad65ff6037"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:03:40.641281 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.641175    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28b51def-e1d3-49ae-b793-dcdaba596e68-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "28b51def-e1d3-49ae-b793-dcdaba596e68" (UID: "28b51def-e1d3-49ae-b793-dcdaba596e68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:03:40.642865 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.642839    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-kube-api-access-749nr" (OuterVolumeSpecName: "kube-api-access-749nr") pod "b87b92da-6c3c-4e8d-93ec-f0ad65ff6037" (UID: "b87b92da-6c3c-4e8d-93ec-f0ad65ff6037"). InnerVolumeSpecName "kube-api-access-749nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:03:40.643099 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.643079    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28b51def-e1d3-49ae-b793-dcdaba596e68-kube-api-access-w2d7w" (OuterVolumeSpecName: "kube-api-access-w2d7w") pod "28b51def-e1d3-49ae-b793-dcdaba596e68" (UID: "28b51def-e1d3-49ae-b793-dcdaba596e68"). InnerVolumeSpecName "kube-api-access-w2d7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:03:40.720675 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.720604    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"]
Aug 04 00:03:40.742343 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.742319    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:03:40.742343 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.742338    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2d7w\" (UniqueName: \"kubernetes.io/projected/28b51def-e1d3-49ae-b793-dcdaba596e68-kube-api-access-w2d7w\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:03:40.742492 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.742348    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b51def-e1d3-49ae-b793-dcdaba596e68-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:03:40.742492 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.742358    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-749nr\" (UniqueName: \"kubernetes.io/projected/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037-kube-api-access-749nr\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:03:40.747357 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.747330    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f9c1edcb-774bb887f4-bqkcd"]
Aug 04 00:03:40.764179 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.764159    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="429dfc4f-41ba-42c4-b59b-dcdc9a9a7395" path="/var/lib/kubelet/pods/429dfc4f-41ba-42c4-b59b-dcdc9a9a7395/volumes"
Aug 04 00:03:40.884752 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.884725    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"]
Aug 04 00:03:40.893434 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.893410    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f9c1edcb-7494dfb8b4-l7ck4"]
Aug 04 00:03:40.922191 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.922160    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"]
Aug 04 00:03:40.952701 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:40.952670    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f9c1edcb-555f47874-pnsdk"]
Aug 04 00:03:42.765020 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:42.764988    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28b51def-e1d3-49ae-b793-dcdaba596e68" path="/var/lib/kubelet/pods/28b51def-e1d3-49ae-b793-dcdaba596e68/volumes"
Aug 04 00:03:42.765490 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:42.765468    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b87b92da-6c3c-4e8d-93ec-f0ad65ff6037" path="/var/lib/kubelet/pods/b87b92da-6c3c-4e8d-93ec-f0ad65ff6037/volumes"
Aug 04 00:03:43.941170 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.941140    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"]
Aug 04 00:03:43.941680 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.941661    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b87b92da-6c3c-4e8d-93ec-f0ad65ff6037" containerName="payload-pre-processing"
Aug 04 00:03:43.941786 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.941683    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b87b92da-6c3c-4e8d-93ec-f0ad65ff6037" containerName="payload-pre-processing"
Aug 04 00:03:43.941786 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.941698    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="429dfc4f-41ba-42c4-b59b-dcdc9a9a7395" containerName="maas-api"
Aug 04 00:03:43.941786 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.941723    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="429dfc4f-41ba-42c4-b59b-dcdc9a9a7395" containerName="maas-api"
Aug 04 00:03:43.941786 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.941755    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28b51def-e1d3-49ae-b793-dcdaba596e68" containerName="payload-processing"
Aug 04 00:03:43.941786 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.941764    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="28b51def-e1d3-49ae-b793-dcdaba596e68" containerName="payload-processing"
Aug 04 00:03:43.942019 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.941853    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="429dfc4f-41ba-42c4-b59b-dcdc9a9a7395" containerName="maas-api"
Aug 04 00:03:43.942019 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.941867    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b87b92da-6c3c-4e8d-93ec-f0ad65ff6037" containerName="payload-pre-processing"
Aug 04 00:03:43.942019 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.941879    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="28b51def-e1d3-49ae-b793-dcdaba596e68" containerName="payload-processing"
Aug 04 00:03:43.946194 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.946175    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:43.948563 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.948546    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-50db8767-serving-cert\""
Aug 04 00:03:43.952951 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:43.952925    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"]
Aug 04 00:03:44.075642 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.075612    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-openshift-service-ca\") pod \"maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:44.075642 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.075643    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm98h\" (UniqueName: \"kubernetes.io/projected/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-kube-api-access-sm98h\") pod \"maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:44.075861 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.075691    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-maas-api-tls\") pod \"maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:44.177086 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.177054    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-maas-api-tls\") pod \"maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:44.177239 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.177118    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-openshift-service-ca\") pod \"maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:44.177239 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.177139    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm98h\" (UniqueName: \"kubernetes.io/projected/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-kube-api-access-sm98h\") pod \"maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:44.177239 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:03:44.177204    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-serving-cert: secret "maas-api-e2e-migrate-50db8767-serving-cert" not found
Aug 04 00:03:44.177342 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:03:44.177274    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-maas-api-tls podName:a6dc35e7-041f-4b7c-bb25-0840618eb3f9 nodeName:}" failed. No retries permitted until 2026-08-04 00:03:44.677256526 +0000 UTC m=+2032.378963171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-maas-api-tls") pod "maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj" (UID: "a6dc35e7-041f-4b7c-bb25-0840618eb3f9") : secret "maas-api-e2e-migrate-50db8767-serving-cert" not found
Aug 04 00:03:44.177747 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.177699    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-openshift-service-ca\") pod \"maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:44.186951 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.186930    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm98h\" (UniqueName: \"kubernetes.io/projected/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-kube-api-access-sm98h\") pod \"maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:44.643365 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.643331    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"]
Aug 04 00:03:44.646991 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.646977    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:03:44.649315 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.649291    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-50db8767-dockercfg-5mbq5\""
Aug 04 00:03:44.649445 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.649291    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-50db8767\""
Aug 04 00:03:44.654051 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.654031    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"]
Aug 04 00:03:44.681370 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.681341    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-maas-api-tls\") pod \"maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:44.684022 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.683999    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-maas-api-tls\") pod \"maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:44.690934 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.690913    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"]
Aug 04 00:03:44.694581 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.694565    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:03:44.701113 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.701095    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"]
Aug 04 00:03:44.781951 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.781921    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v57s2\" (UniqueName: \"kubernetes.io/projected/75de96bd-ada9-4a74-bbb3-e15934f4e21e-kube-api-access-v57s2\") pod \"payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4\" (UID: \"75de96bd-ada9-4a74-bbb3-e15934f4e21e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:03:44.781951 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.781953    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf91a70-ad84-450d-8c35-c7a20af0e36b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9\" (UID: \"acf91a70-ad84-450d-8c35-c7a20af0e36b\") " pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:03:44.782131 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.781984    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9bdh\" (UniqueName: \"kubernetes.io/projected/acf91a70-ad84-450d-8c35-c7a20af0e36b-kube-api-access-k9bdh\") pod \"payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9\" (UID: \"acf91a70-ad84-450d-8c35-c7a20af0e36b\") " pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:03:44.782131 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.782044    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75de96bd-ada9-4a74-bbb3-e15934f4e21e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4\" (UID: \"75de96bd-ada9-4a74-bbb3-e15934f4e21e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:03:44.858247 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.858185    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:44.883404 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.883359    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75de96bd-ada9-4a74-bbb3-e15934f4e21e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4\" (UID: \"75de96bd-ada9-4a74-bbb3-e15934f4e21e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:03:44.883584 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.883554    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v57s2\" (UniqueName: \"kubernetes.io/projected/75de96bd-ada9-4a74-bbb3-e15934f4e21e-kube-api-access-v57s2\") pod \"payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4\" (UID: \"75de96bd-ada9-4a74-bbb3-e15934f4e21e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:03:44.883723 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.883601    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf91a70-ad84-450d-8c35-c7a20af0e36b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9\" (UID: \"acf91a70-ad84-450d-8c35-c7a20af0e36b\") " pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:03:44.883723 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.883640    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9bdh\" (UniqueName: \"kubernetes.io/projected/acf91a70-ad84-450d-8c35-c7a20af0e36b-kube-api-access-k9bdh\") pod \"payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9\" (UID: \"acf91a70-ad84-450d-8c35-c7a20af0e36b\") " pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:03:44.884193 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.884170    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75de96bd-ada9-4a74-bbb3-e15934f4e21e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4\" (UID: \"75de96bd-ada9-4a74-bbb3-e15934f4e21e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:03:44.884419 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.884395    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf91a70-ad84-450d-8c35-c7a20af0e36b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9\" (UID: \"acf91a70-ad84-450d-8c35-c7a20af0e36b\") " pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:03:44.891832 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.891808    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9bdh\" (UniqueName: \"kubernetes.io/projected/acf91a70-ad84-450d-8c35-c7a20af0e36b-kube-api-access-k9bdh\") pod \"payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9\" (UID: \"acf91a70-ad84-450d-8c35-c7a20af0e36b\") " pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:03:44.891919 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.891888    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v57s2\" (UniqueName: \"kubernetes.io/projected/75de96bd-ada9-4a74-bbb3-e15934f4e21e-kube-api-access-v57s2\") pod \"payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4\" (UID: \"75de96bd-ada9-4a74-bbb3-e15934f4e21e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:03:44.958767 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.958742    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:03:44.993501 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:44.993464    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"]
Aug 04 00:03:44.997151 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:03:44.997126    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6dc35e7_041f_4b7c_bb25_0840618eb3f9.slice/crio-6aca4ac9378aa2f40a61f8f00d5d6839b02a33491a50552d7d6656860a765702 WatchSource:0}: Error finding container 6aca4ac9378aa2f40a61f8f00d5d6839b02a33491a50552d7d6656860a765702: Status 404 returned error can't find the container with id 6aca4ac9378aa2f40a61f8f00d5d6839b02a33491a50552d7d6656860a765702
Aug 04 00:03:45.006118 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.006055    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:03:45.107514 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.107369    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"]
Aug 04 00:03:45.110170 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:03:45.110140    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75de96bd_ada9_4a74_bbb3_e15934f4e21e.slice/crio-a435dc20ad7be44f72e2b237c4c5ffa3d686f7e283436159ab759adf3202409a WatchSource:0}: Error finding container a435dc20ad7be44f72e2b237c4c5ffa3d686f7e283436159ab759adf3202409a: Status 404 returned error can't find the container with id a435dc20ad7be44f72e2b237c4c5ffa3d686f7e283436159ab759adf3202409a
Aug 04 00:03:45.152259 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.152237    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"]
Aug 04 00:03:45.573451 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.573405    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4" event={"ID":"75de96bd-ada9-4a74-bbb3-e15934f4e21e","Type":"ContainerStarted","Data":"753b04dd62efc9843839147c2d6883f3991525a345ecb9e65388ff4527bbbe9b"}
Aug 04 00:03:45.573614 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.573467    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4" event={"ID":"75de96bd-ada9-4a74-bbb3-e15934f4e21e","Type":"ContainerStarted","Data":"a435dc20ad7be44f72e2b237c4c5ffa3d686f7e283436159ab759adf3202409a"}
Aug 04 00:03:45.573614 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.573573    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:03:45.575047 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.575012    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj" event={"ID":"a6dc35e7-041f-4b7c-bb25-0840618eb3f9","Type":"ContainerStarted","Data":"839aa38bc2cfd5c7e920ee3d5894b34bdd88684d90b1d05255b01e4c5b6664c4"}
Aug 04 00:03:45.575175 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.575060    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:45.575175 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.575081    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj" event={"ID":"a6dc35e7-041f-4b7c-bb25-0840618eb3f9","Type":"ContainerStarted","Data":"6aca4ac9378aa2f40a61f8f00d5d6839b02a33491a50552d7d6656860a765702"}
Aug 04 00:03:45.576541 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.576507    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9" event={"ID":"acf91a70-ad84-450d-8c35-c7a20af0e36b","Type":"ContainerStarted","Data":"95e0271c50770ecbe3394fd4e63ac15695a938395ff37f80f7351e00054455ad"}
Aug 04 00:03:45.576656 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.576557    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9" event={"ID":"acf91a70-ad84-450d-8c35-c7a20af0e36b","Type":"ContainerStarted","Data":"fad367e35bf16b2f45f5b1bb9385ad83298112951872c72388550a4c02f23741"}
Aug 04 00:03:45.576656 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.576587    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:03:45.587280 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.587243    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4" podStartSLOduration=1.587231064 podStartE2EDuration="1.587231064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:03:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:45.585742385 +0000 UTC m=+2033.287449053" watchObservedRunningTime="2026-08-04 00:03:45.587231064 +0000 UTC m=+2033.288937726"
Aug 04 00:03:45.599409 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.599371    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9" podStartSLOduration=1.599358841 podStartE2EDuration="1.599358841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:03:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:45.59865351 +0000 UTC m=+2033.300360176" watchObservedRunningTime="2026-08-04 00:03:45.599358841 +0000 UTC m=+2033.301065504"
Aug 04 00:03:45.614105 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:45.614055    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj" podStartSLOduration=2.245912546 podStartE2EDuration="2.614040138s" totalImagesPullingTime="368.127592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:03:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:45.612767456 +0000 UTC m=+2033.314474114" watchObservedRunningTime="2026-08-04 00:03:45.614040138 +0000 UTC m=+2033.315746803"
Aug 04 00:03:51.587999 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:51.587964    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:03:56.584567 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:56.584536    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:03:56.585037 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:03:56.584697    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:04:14.479651 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.479572    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"]
Aug 04 00:04:14.480209 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.479872    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj" podUID="a6dc35e7-041f-4b7c-bb25-0840618eb3f9" containerName="maas-api" containerID="cri-o://839aa38bc2cfd5c7e920ee3d5894b34bdd88684d90b1d05255b01e4c5b6664c4" gracePeriod=30
Aug 04 00:04:14.528806 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.528780    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"]
Aug 04 00:04:14.529039 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.529013    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9" podUID="acf91a70-ad84-450d-8c35-c7a20af0e36b" containerName="payload-processing" containerID="cri-o://95e0271c50770ecbe3394fd4e63ac15695a938395ff37f80f7351e00054455ad" gracePeriod=30
Aug 04 00:04:14.539275 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.539252    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"]
Aug 04 00:04:14.539521 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.539473    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4" podUID="75de96bd-ada9-4a74-bbb3-e15934f4e21e" containerName="payload-pre-processing" containerID="cri-o://753b04dd62efc9843839147c2d6883f3991525a345ecb9e65388ff4527bbbe9b" gracePeriod=30
Aug 04 00:04:14.735838 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.735769    2553 generic.go:356] "Generic (PLEG): container finished" podID="75de96bd-ada9-4a74-bbb3-e15934f4e21e" containerID="753b04dd62efc9843839147c2d6883f3991525a345ecb9e65388ff4527bbbe9b" exitCode=0
Aug 04 00:04:14.735954 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.735868    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4" event={"ID":"75de96bd-ada9-4a74-bbb3-e15934f4e21e","Type":"ContainerDied","Data":"753b04dd62efc9843839147c2d6883f3991525a345ecb9e65388ff4527bbbe9b"}
Aug 04 00:04:14.738040 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.738016    2553 generic.go:356] "Generic (PLEG): container finished" podID="a6dc35e7-041f-4b7c-bb25-0840618eb3f9" containerID="839aa38bc2cfd5c7e920ee3d5894b34bdd88684d90b1d05255b01e4c5b6664c4" exitCode=0
Aug 04 00:04:14.738150 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.738063    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj" event={"ID":"a6dc35e7-041f-4b7c-bb25-0840618eb3f9","Type":"ContainerDied","Data":"839aa38bc2cfd5c7e920ee3d5894b34bdd88684d90b1d05255b01e4c5b6664c4"}
Aug 04 00:04:14.739628 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.739585    2553 generic.go:356] "Generic (PLEG): container finished" podID="acf91a70-ad84-450d-8c35-c7a20af0e36b" containerID="95e0271c50770ecbe3394fd4e63ac15695a938395ff37f80f7351e00054455ad" exitCode=0
Aug 04 00:04:14.739775 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.739640    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9" event={"ID":"acf91a70-ad84-450d-8c35-c7a20af0e36b","Type":"ContainerDied","Data":"95e0271c50770ecbe3394fd4e63ac15695a938395ff37f80f7351e00054455ad"}
Aug 04 00:04:14.798442 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.798409    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:04:14.829351 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.829325    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:04:14.834680 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.834661    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:04:14.837176 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.837153    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-maas-api-tls\") pod \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") "
Aug 04 00:04:14.837241 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.837200    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-openshift-service-ca\") pod \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") "
Aug 04 00:04:14.837390 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.837373    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm98h\" (UniqueName: \"kubernetes.io/projected/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-kube-api-access-sm98h\") pod \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\" (UID: \"a6dc35e7-041f-4b7c-bb25-0840618eb3f9\") "
Aug 04 00:04:14.837553 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.837525    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a6dc35e7-041f-4b7c-bb25-0840618eb3f9" (UID: "a6dc35e7-041f-4b7c-bb25-0840618eb3f9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:04:14.837815 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.837792    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:14.839234 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.839210    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a6dc35e7-041f-4b7c-bb25-0840618eb3f9" (UID: "a6dc35e7-041f-4b7c-bb25-0840618eb3f9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:04:14.839473 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.839458    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-kube-api-access-sm98h" (OuterVolumeSpecName: "kube-api-access-sm98h") pod "a6dc35e7-041f-4b7c-bb25-0840618eb3f9" (UID: "a6dc35e7-041f-4b7c-bb25-0840618eb3f9"). InnerVolumeSpecName "kube-api-access-sm98h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:04:14.938602 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.938574    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9bdh\" (UniqueName: \"kubernetes.io/projected/acf91a70-ad84-450d-8c35-c7a20af0e36b-kube-api-access-k9bdh\") pod \"acf91a70-ad84-450d-8c35-c7a20af0e36b\" (UID: \"acf91a70-ad84-450d-8c35-c7a20af0e36b\") "
Aug 04 00:04:14.938793 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.938655    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75de96bd-ada9-4a74-bbb3-e15934f4e21e-plugins-config-volume\") pod \"75de96bd-ada9-4a74-bbb3-e15934f4e21e\" (UID: \"75de96bd-ada9-4a74-bbb3-e15934f4e21e\") "
Aug 04 00:04:14.938793 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.938678    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf91a70-ad84-450d-8c35-c7a20af0e36b-plugins-config-volume\") pod \"acf91a70-ad84-450d-8c35-c7a20af0e36b\" (UID: \"acf91a70-ad84-450d-8c35-c7a20af0e36b\") "
Aug 04 00:04:14.938793 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.938702    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v57s2\" (UniqueName: \"kubernetes.io/projected/75de96bd-ada9-4a74-bbb3-e15934f4e21e-kube-api-access-v57s2\") pod \"75de96bd-ada9-4a74-bbb3-e15934f4e21e\" (UID: \"75de96bd-ada9-4a74-bbb3-e15934f4e21e\") "
Aug 04 00:04:14.939054 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.939028    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf91a70-ad84-450d-8c35-c7a20af0e36b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "acf91a70-ad84-450d-8c35-c7a20af0e36b" (UID: "acf91a70-ad84-450d-8c35-c7a20af0e36b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:04:14.939054 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.939050    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:14.939184 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.939070    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm98h\" (UniqueName: \"kubernetes.io/projected/a6dc35e7-041f-4b7c-bb25-0840618eb3f9-kube-api-access-sm98h\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:14.939184 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.939048    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75de96bd-ada9-4a74-bbb3-e15934f4e21e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "75de96bd-ada9-4a74-bbb3-e15934f4e21e" (UID: "75de96bd-ada9-4a74-bbb3-e15934f4e21e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:04:14.940863 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.940844    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75de96bd-ada9-4a74-bbb3-e15934f4e21e-kube-api-access-v57s2" (OuterVolumeSpecName: "kube-api-access-v57s2") pod "75de96bd-ada9-4a74-bbb3-e15934f4e21e" (UID: "75de96bd-ada9-4a74-bbb3-e15934f4e21e"). InnerVolumeSpecName "kube-api-access-v57s2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:04:14.940972 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:14.940955    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf91a70-ad84-450d-8c35-c7a20af0e36b-kube-api-access-k9bdh" (OuterVolumeSpecName: "kube-api-access-k9bdh") pod "acf91a70-ad84-450d-8c35-c7a20af0e36b" (UID: "acf91a70-ad84-450d-8c35-c7a20af0e36b"). InnerVolumeSpecName "kube-api-access-k9bdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:04:15.039873 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.039815    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9bdh\" (UniqueName: \"kubernetes.io/projected/acf91a70-ad84-450d-8c35-c7a20af0e36b-kube-api-access-k9bdh\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:15.039873 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.039837    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75de96bd-ada9-4a74-bbb3-e15934f4e21e-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:15.039873 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.039846    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf91a70-ad84-450d-8c35-c7a20af0e36b-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:15.039873 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.039854    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v57s2\" (UniqueName: \"kubernetes.io/projected/75de96bd-ada9-4a74-bbb3-e15934f4e21e-kube-api-access-v57s2\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:15.745583 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.745507    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"
Aug 04 00:04:15.745583 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.745519    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4" event={"ID":"75de96bd-ada9-4a74-bbb3-e15934f4e21e","Type":"ContainerDied","Data":"a435dc20ad7be44f72e2b237c4c5ffa3d686f7e283436159ab759adf3202409a"}
Aug 04 00:04:15.745583 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.745582    2553 scope.go:117] "RemoveContainer" containerID="753b04dd62efc9843839147c2d6883f3991525a345ecb9e65388ff4527bbbe9b"
Aug 04 00:04:15.746979 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.746947    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj" event={"ID":"a6dc35e7-041f-4b7c-bb25-0840618eb3f9","Type":"ContainerDied","Data":"6aca4ac9378aa2f40a61f8f00d5d6839b02a33491a50552d7d6656860a765702"}
Aug 04 00:04:15.747106 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.746981    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"
Aug 04 00:04:15.748877 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.748852    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9" event={"ID":"acf91a70-ad84-450d-8c35-c7a20af0e36b","Type":"ContainerDied","Data":"fad367e35bf16b2f45f5b1bb9385ad83298112951872c72388550a4c02f23741"}
Aug 04 00:04:15.748877 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.748871    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"
Aug 04 00:04:15.755748 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.755725    2553 scope.go:117] "RemoveContainer" containerID="839aa38bc2cfd5c7e920ee3d5894b34bdd88684d90b1d05255b01e4c5b6664c4"
Aug 04 00:04:15.763677 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.763663    2553 scope.go:117] "RemoveContainer" containerID="95e0271c50770ecbe3394fd4e63ac15695a938395ff37f80f7351e00054455ad"
Aug 04 00:04:15.771021 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.771003    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"]
Aug 04 00:04:15.774277 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.774258    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-50db8767-584c665496-lmzl4"]
Aug 04 00:04:15.784416 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.784395    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"]
Aug 04 00:04:15.789577 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.789557    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-50db8767-6c56d7f6f7-72jf9"]
Aug 04 00:04:15.799801 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.799783    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"]
Aug 04 00:04:15.803753 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:15.803734    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-50db8767-788bb7d9f-7ffdj"]
Aug 04 00:04:16.763195 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:16.763168    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75de96bd-ada9-4a74-bbb3-e15934f4e21e" path="/var/lib/kubelet/pods/75de96bd-ada9-4a74-bbb3-e15934f4e21e/volumes"
Aug 04 00:04:16.763549 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:16.763512    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6dc35e7-041f-4b7c-bb25-0840618eb3f9" path="/var/lib/kubelet/pods/a6dc35e7-041f-4b7c-bb25-0840618eb3f9/volumes"
Aug 04 00:04:16.763829 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:16.763815    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acf91a70-ad84-450d-8c35-c7a20af0e36b" path="/var/lib/kubelet/pods/acf91a70-ad84-450d-8c35-c7a20af0e36b/volumes"
Aug 04 00:04:19.177051 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.177015    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"]
Aug 04 00:04:19.177580 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.177562    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf91a70-ad84-450d-8c35-c7a20af0e36b" containerName="payload-processing"
Aug 04 00:04:19.177630 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.177586    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf91a70-ad84-450d-8c35-c7a20af0e36b" containerName="payload-processing"
Aug 04 00:04:19.177630 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.177605    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6dc35e7-041f-4b7c-bb25-0840618eb3f9" containerName="maas-api"
Aug 04 00:04:19.177630 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.177614    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6dc35e7-041f-4b7c-bb25-0840618eb3f9" containerName="maas-api"
Aug 04 00:04:19.177757 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.177632    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75de96bd-ada9-4a74-bbb3-e15934f4e21e" containerName="payload-pre-processing"
Aug 04 00:04:19.177757 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.177642    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="75de96bd-ada9-4a74-bbb3-e15934f4e21e" containerName="payload-pre-processing"
Aug 04 00:04:19.177840 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.177768    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="75de96bd-ada9-4a74-bbb3-e15934f4e21e" containerName="payload-pre-processing"
Aug 04 00:04:19.177840 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.177784    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="acf91a70-ad84-450d-8c35-c7a20af0e36b" containerName="payload-processing"
Aug 04 00:04:19.177840 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.177797    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6dc35e7-041f-4b7c-bb25-0840618eb3f9" containerName="maas-api"
Aug 04 00:04:19.183104 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.183084    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:19.185544 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.185524    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-3113ce29-serving-cert\""
Aug 04 00:04:19.192462 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.192438    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"]
Aug 04 00:04:19.277644 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.277620    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc1120e3-8963-40dd-bfb4-7d2816e221aa-maas-api-tls\") pod \"maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:19.277807 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.277679    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xspc8\" (UniqueName: \"kubernetes.io/projected/dc1120e3-8963-40dd-bfb4-7d2816e221aa-kube-api-access-xspc8\") pod \"maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:19.277863 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.277795    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1120e3-8963-40dd-bfb4-7d2816e221aa-openshift-service-ca\") pod \"maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:19.378996 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.378972    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1120e3-8963-40dd-bfb4-7d2816e221aa-openshift-service-ca\") pod \"maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:19.379101 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.379019    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc1120e3-8963-40dd-bfb4-7d2816e221aa-maas-api-tls\") pod \"maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:19.379101 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.379051    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xspc8\" (UniqueName: \"kubernetes.io/projected/dc1120e3-8963-40dd-bfb4-7d2816e221aa-kube-api-access-xspc8\") pod \"maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:19.379671 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.379653    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1120e3-8963-40dd-bfb4-7d2816e221aa-openshift-service-ca\") pod \"maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:19.381565 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.381544    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc1120e3-8963-40dd-bfb4-7d2816e221aa-maas-api-tls\") pod \"maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:19.386652 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.386626    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xspc8\" (UniqueName: \"kubernetes.io/projected/dc1120e3-8963-40dd-bfb4-7d2816e221aa-kube-api-access-xspc8\") pod \"maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:19.497568 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.497502    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:19.635679 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.635654    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"]
Aug 04 00:04:19.637604 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:04:19.637580    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1120e3_8963_40dd_bfb4_7d2816e221aa.slice/crio-f90c455de1f51d7602d314578ed472ef0678668637f9c616bb1a5c45e9568ef6 WatchSource:0}: Error finding container f90c455de1f51d7602d314578ed472ef0678668637f9c616bb1a5c45e9568ef6: Status 404 returned error can't find the container with id f90c455de1f51d7602d314578ed472ef0678668637f9c616bb1a5c45e9568ef6
Aug 04 00:04:19.774986 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.774904    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8" event={"ID":"dc1120e3-8963-40dd-bfb4-7d2816e221aa","Type":"ContainerStarted","Data":"f90c455de1f51d7602d314578ed472ef0678668637f9c616bb1a5c45e9568ef6"}
Aug 04 00:04:19.903979 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.903950    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"]
Aug 04 00:04:19.908540 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.908524    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:19.911976 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.911959    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3113ce29\""
Aug 04 00:04:19.912071 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.911963    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-3113ce29-dockercfg-7ds8m\""
Aug 04 00:04:19.919919 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.919895    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"]
Aug 04 00:04:19.965301 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.965276    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"]
Aug 04 00:04:19.968596 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.968577    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:19.980509 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.980489    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"]
Aug 04 00:04:19.984834 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.984815    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a5da00d-6661-4e90-8cb9-82d9f659676f-plugins-config-volume\") pod \"payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq\" (UID: \"0a5da00d-6661-4e90-8cb9-82d9f659676f\") " pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:19.984928 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:19.984869    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwxj8\" (UniqueName: \"kubernetes.io/projected/0a5da00d-6661-4e90-8cb9-82d9f659676f-kube-api-access-hwxj8\") pod \"payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq\" (UID: \"0a5da00d-6661-4e90-8cb9-82d9f659676f\") " pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:20.085458 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.085388    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwxj8\" (UniqueName: \"kubernetes.io/projected/0a5da00d-6661-4e90-8cb9-82d9f659676f-kube-api-access-hwxj8\") pod \"payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq\" (UID: \"0a5da00d-6661-4e90-8cb9-82d9f659676f\") " pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:20.085458 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.085432    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kmdn\" (UniqueName: \"kubernetes.io/projected/e57593fb-3313-4c89-982e-bdc125437a71-kube-api-access-4kmdn\") pod \"payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk\" (UID: \"e57593fb-3313-4c89-982e-bdc125437a71\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:20.085624 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.085467    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e57593fb-3313-4c89-982e-bdc125437a71-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk\" (UID: \"e57593fb-3313-4c89-982e-bdc125437a71\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:20.085624 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.085504    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a5da00d-6661-4e90-8cb9-82d9f659676f-plugins-config-volume\") pod \"payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq\" (UID: \"0a5da00d-6661-4e90-8cb9-82d9f659676f\") " pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:20.086145 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.086129    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a5da00d-6661-4e90-8cb9-82d9f659676f-plugins-config-volume\") pod \"payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq\" (UID: \"0a5da00d-6661-4e90-8cb9-82d9f659676f\") " pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:20.093989 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.093969    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwxj8\" (UniqueName: \"kubernetes.io/projected/0a5da00d-6661-4e90-8cb9-82d9f659676f-kube-api-access-hwxj8\") pod \"payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq\" (UID: \"0a5da00d-6661-4e90-8cb9-82d9f659676f\") " pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:20.186501 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.186474    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e57593fb-3313-4c89-982e-bdc125437a71-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk\" (UID: \"e57593fb-3313-4c89-982e-bdc125437a71\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:20.186936 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.186545    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kmdn\" (UniqueName: \"kubernetes.io/projected/e57593fb-3313-4c89-982e-bdc125437a71-kube-api-access-4kmdn\") pod \"payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk\" (UID: \"e57593fb-3313-4c89-982e-bdc125437a71\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:20.187278 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.187256    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e57593fb-3313-4c89-982e-bdc125437a71-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk\" (UID: \"e57593fb-3313-4c89-982e-bdc125437a71\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:20.194308 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.194287    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kmdn\" (UniqueName: \"kubernetes.io/projected/e57593fb-3313-4c89-982e-bdc125437a71-kube-api-access-4kmdn\") pod \"payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk\" (UID: \"e57593fb-3313-4c89-982e-bdc125437a71\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:20.219774 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.219755    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:20.278263 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.278238    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:20.361401 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.361372    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"]
Aug 04 00:04:20.362668 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:04:20.362642    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a5da00d_6661_4e90_8cb9_82d9f659676f.slice/crio-8bd92646cf7ce81b0d57353476cdeff130a08ed5bcd916bb24502469d3c96b19 WatchSource:0}: Error finding container 8bd92646cf7ce81b0d57353476cdeff130a08ed5bcd916bb24502469d3c96b19: Status 404 returned error can't find the container with id 8bd92646cf7ce81b0d57353476cdeff130a08ed5bcd916bb24502469d3c96b19
Aug 04 00:04:20.427375 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.427330    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"]
Aug 04 00:04:20.429935 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:04:20.429904    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode57593fb_3313_4c89_982e_bdc125437a71.slice/crio-d63a9eba51432a9e8ddf380c35cdff011d2097b1d684305ccfd1a47c923603e5 WatchSource:0}: Error finding container d63a9eba51432a9e8ddf380c35cdff011d2097b1d684305ccfd1a47c923603e5: Status 404 returned error can't find the container with id d63a9eba51432a9e8ddf380c35cdff011d2097b1d684305ccfd1a47c923603e5
Aug 04 00:04:20.781945 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.781906    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq" event={"ID":"0a5da00d-6661-4e90-8cb9-82d9f659676f","Type":"ContainerStarted","Data":"18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f"}
Aug 04 00:04:20.782132 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.781965    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq" event={"ID":"0a5da00d-6661-4e90-8cb9-82d9f659676f","Type":"ContainerStarted","Data":"8bd92646cf7ce81b0d57353476cdeff130a08ed5bcd916bb24502469d3c96b19"}
Aug 04 00:04:20.782132 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.782008    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:20.783476 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.783449    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8" event={"ID":"dc1120e3-8963-40dd-bfb4-7d2816e221aa","Type":"ContainerStarted","Data":"1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa"}
Aug 04 00:04:20.783605 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.783503    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:20.784941 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.784907    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk" event={"ID":"e57593fb-3313-4c89-982e-bdc125437a71","Type":"ContainerStarted","Data":"4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4"}
Aug 04 00:04:20.784941 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.784947    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk" event={"ID":"e57593fb-3313-4c89-982e-bdc125437a71","Type":"ContainerStarted","Data":"d63a9eba51432a9e8ddf380c35cdff011d2097b1d684305ccfd1a47c923603e5"}
Aug 04 00:04:20.784941 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.784968    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:20.798362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.798322    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq" podStartSLOduration=1.798310028 podStartE2EDuration="1.798310028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:04:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:04:20.79625239 +0000 UTC m=+2068.497959053" watchObservedRunningTime="2026-08-04 00:04:20.798310028 +0000 UTC m=+2068.500016669"
Aug 04 00:04:20.810403 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.810365    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk" podStartSLOduration=1.8103516069999999 podStartE2EDuration="1.810351607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:04:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:04:20.809646519 +0000 UTC m=+2068.511353183" watchObservedRunningTime="2026-08-04 00:04:20.810351607 +0000 UTC m=+2068.512058270"
Aug 04 00:04:20.826168 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:20.826133    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8" podStartSLOduration=1.210964239 podStartE2EDuration="1.826121223s" totalImagesPullingTime="615.156984ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:04:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:04:20.824626634 +0000 UTC m=+2068.526333309" watchObservedRunningTime="2026-08-04 00:04:20.826121223 +0000 UTC m=+2068.527827886"
Aug 04 00:04:26.795416 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:26.795389    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:31.791538 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:31.791509    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:31.792049 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:31.791583    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:49.476383 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.476350    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"]
Aug 04 00:04:49.476998 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.476619    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8" podUID="dc1120e3-8963-40dd-bfb4-7d2816e221aa" containerName="maas-api" containerID="cri-o://1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa" gracePeriod=30
Aug 04 00:04:49.530300 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.530268    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"]
Aug 04 00:04:49.530528 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.530504    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq" podUID="0a5da00d-6661-4e90-8cb9-82d9f659676f" containerName="payload-processing" containerID="cri-o://18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f" gracePeriod=30
Aug 04 00:04:49.536602 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.536580    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"]
Aug 04 00:04:49.537024 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.536984    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk" podUID="e57593fb-3313-4c89-982e-bdc125437a71" containerName="payload-pre-processing" containerID="cri-o://4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4" gracePeriod=30
Aug 04 00:04:49.809051 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.809027    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:49.836008 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.835983    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:49.839542 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.839515    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:49.927929 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.927888    2553 generic.go:356] "Generic (PLEG): container finished" podID="dc1120e3-8963-40dd-bfb4-7d2816e221aa" containerID="1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa" exitCode=0
Aug 04 00:04:49.928114 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.927949    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"
Aug 04 00:04:49.928114 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.927965    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8" event={"ID":"dc1120e3-8963-40dd-bfb4-7d2816e221aa","Type":"ContainerDied","Data":"1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa"}
Aug 04 00:04:49.928114 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.928060    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8" event={"ID":"dc1120e3-8963-40dd-bfb4-7d2816e221aa","Type":"ContainerDied","Data":"f90c455de1f51d7602d314578ed472ef0678668637f9c616bb1a5c45e9568ef6"}
Aug 04 00:04:49.928114 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.928095    2553 scope.go:117] "RemoveContainer" containerID="1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa"
Aug 04 00:04:49.929202 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.929183    2553 generic.go:356] "Generic (PLEG): container finished" podID="e57593fb-3313-4c89-982e-bdc125437a71" containerID="4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4" exitCode=0
Aug 04 00:04:49.929314 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.929297    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"
Aug 04 00:04:49.929420 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.929295    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk" event={"ID":"e57593fb-3313-4c89-982e-bdc125437a71","Type":"ContainerDied","Data":"4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4"}
Aug 04 00:04:49.929506 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.929453    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk" event={"ID":"e57593fb-3313-4c89-982e-bdc125437a71","Type":"ContainerDied","Data":"d63a9eba51432a9e8ddf380c35cdff011d2097b1d684305ccfd1a47c923603e5"}
Aug 04 00:04:49.930824 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.930790    2553 generic.go:356] "Generic (PLEG): container finished" podID="0a5da00d-6661-4e90-8cb9-82d9f659676f" containerID="18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f" exitCode=0
Aug 04 00:04:49.931103 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.930828    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq" event={"ID":"0a5da00d-6661-4e90-8cb9-82d9f659676f","Type":"ContainerDied","Data":"18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f"}
Aug 04 00:04:49.931103 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.931107    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq" event={"ID":"0a5da00d-6661-4e90-8cb9-82d9f659676f","Type":"ContainerDied","Data":"8bd92646cf7ce81b0d57353476cdeff130a08ed5bcd916bb24502469d3c96b19"}
Aug 04 00:04:49.931103 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.930874    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"
Aug 04 00:04:49.936162 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.936137    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1120e3-8963-40dd-bfb4-7d2816e221aa-openshift-service-ca\") pod \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") "
Aug 04 00:04:49.936282 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.936178    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e57593fb-3313-4c89-982e-bdc125437a71-plugins-config-volume\") pod \"e57593fb-3313-4c89-982e-bdc125437a71\" (UID: \"e57593fb-3313-4c89-982e-bdc125437a71\") "
Aug 04 00:04:49.936282 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.936210    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kmdn\" (UniqueName: \"kubernetes.io/projected/e57593fb-3313-4c89-982e-bdc125437a71-kube-api-access-4kmdn\") pod \"e57593fb-3313-4c89-982e-bdc125437a71\" (UID: \"e57593fb-3313-4c89-982e-bdc125437a71\") "
Aug 04 00:04:49.936282 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.936252    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc1120e3-8963-40dd-bfb4-7d2816e221aa-maas-api-tls\") pod \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") "
Aug 04 00:04:49.936456 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.936303    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a5da00d-6661-4e90-8cb9-82d9f659676f-plugins-config-volume\") pod \"0a5da00d-6661-4e90-8cb9-82d9f659676f\" (UID: \"0a5da00d-6661-4e90-8cb9-82d9f659676f\") "
Aug 04 00:04:49.936456 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.936375    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xspc8\" (UniqueName: \"kubernetes.io/projected/dc1120e3-8963-40dd-bfb4-7d2816e221aa-kube-api-access-xspc8\") pod \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\" (UID: \"dc1120e3-8963-40dd-bfb4-7d2816e221aa\") "
Aug 04 00:04:49.936456 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.936403    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwxj8\" (UniqueName: \"kubernetes.io/projected/0a5da00d-6661-4e90-8cb9-82d9f659676f-kube-api-access-hwxj8\") pod \"0a5da00d-6661-4e90-8cb9-82d9f659676f\" (UID: \"0a5da00d-6661-4e90-8cb9-82d9f659676f\") "
Aug 04 00:04:49.937236 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.937207    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc1120e3-8963-40dd-bfb4-7d2816e221aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dc1120e3-8963-40dd-bfb4-7d2816e221aa" (UID: "dc1120e3-8963-40dd-bfb4-7d2816e221aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:04:49.937432 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.937409    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e57593fb-3313-4c89-982e-bdc125437a71-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e57593fb-3313-4c89-982e-bdc125437a71" (UID: "e57593fb-3313-4c89-982e-bdc125437a71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:04:49.938031 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.937853    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a5da00d-6661-4e90-8cb9-82d9f659676f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a5da00d-6661-4e90-8cb9-82d9f659676f" (UID: "0a5da00d-6661-4e90-8cb9-82d9f659676f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:04:49.938263 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.938201    2553 scope.go:117] "RemoveContainer" containerID="1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa"
Aug 04 00:04:49.938956 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:04:49.938912    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa\": container with ID starting with 1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa not found: ID does not exist" containerID="1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa"
Aug 04 00:04:49.939057 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.938948    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa"} err="failed to get container status \"1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa\": rpc error: code = NotFound desc = could not find container \"1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa\": container with ID starting with 1a1c257ea0b730a6aeb26e150de23e59e6e8cc4cf4a561ca63b582f02be51cfa not found: ID does not exist"
Aug 04 00:04:49.939057 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.938972    2553 scope.go:117] "RemoveContainer" containerID="4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4"
Aug 04 00:04:49.939585 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.939559    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e57593fb-3313-4c89-982e-bdc125437a71-kube-api-access-4kmdn" (OuterVolumeSpecName: "kube-api-access-4kmdn") pod "e57593fb-3313-4c89-982e-bdc125437a71" (UID: "e57593fb-3313-4c89-982e-bdc125437a71"). InnerVolumeSpecName "kube-api-access-4kmdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:04:49.939697 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.939607    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a5da00d-6661-4e90-8cb9-82d9f659676f-kube-api-access-hwxj8" (OuterVolumeSpecName: "kube-api-access-hwxj8") pod "0a5da00d-6661-4e90-8cb9-82d9f659676f" (UID: "0a5da00d-6661-4e90-8cb9-82d9f659676f"). InnerVolumeSpecName "kube-api-access-hwxj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:04:49.939868 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.939843    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc1120e3-8963-40dd-bfb4-7d2816e221aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dc1120e3-8963-40dd-bfb4-7d2816e221aa" (UID: "dc1120e3-8963-40dd-bfb4-7d2816e221aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:04:49.940567 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.940548    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc1120e3-8963-40dd-bfb4-7d2816e221aa-kube-api-access-xspc8" (OuterVolumeSpecName: "kube-api-access-xspc8") pod "dc1120e3-8963-40dd-bfb4-7d2816e221aa" (UID: "dc1120e3-8963-40dd-bfb4-7d2816e221aa"). InnerVolumeSpecName "kube-api-access-xspc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:04:49.946673 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.946658    2553 scope.go:117] "RemoveContainer" containerID="4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4"
Aug 04 00:04:49.946911 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:04:49.946892    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4\": container with ID starting with 4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4 not found: ID does not exist" containerID="4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4"
Aug 04 00:04:49.946993 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.946920    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4"} err="failed to get container status \"4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4\": rpc error: code = NotFound desc = could not find container \"4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4\": container with ID starting with 4d9ee93e4eea9adf9714b17cdc1b8a2bccfa5852e72a0e358feabfbb38936db4 not found: ID does not exist"
Aug 04 00:04:49.946993 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.946942    2553 scope.go:117] "RemoveContainer" containerID="18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f"
Aug 04 00:04:49.954001 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.953983    2553 scope.go:117] "RemoveContainer" containerID="18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f"
Aug 04 00:04:49.954252 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:04:49.954236    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f\": container with ID starting with 18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f not found: ID does not exist" containerID="18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f"
Aug 04 00:04:49.954300 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:49.954259    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f"} err="failed to get container status \"18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f\": rpc error: code = NotFound desc = could not find container \"18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f\": container with ID starting with 18883fdd3cf7fbba55b64579a0e52a08f4acf67751328d8e723dbf0a36d3173f not found: ID does not exist"
Aug 04 00:04:50.037863 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.037795    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a5da00d-6661-4e90-8cb9-82d9f659676f-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:50.037863 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.037819    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xspc8\" (UniqueName: \"kubernetes.io/projected/dc1120e3-8963-40dd-bfb4-7d2816e221aa-kube-api-access-xspc8\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:50.037863 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.037829    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwxj8\" (UniqueName: \"kubernetes.io/projected/0a5da00d-6661-4e90-8cb9-82d9f659676f-kube-api-access-hwxj8\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:50.037863 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.037837    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1120e3-8963-40dd-bfb4-7d2816e221aa-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:50.037863 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.037845    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e57593fb-3313-4c89-982e-bdc125437a71-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:50.037863 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.037853    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kmdn\" (UniqueName: \"kubernetes.io/projected/e57593fb-3313-4c89-982e-bdc125437a71-kube-api-access-4kmdn\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:50.037863 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.037862    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc1120e3-8963-40dd-bfb4-7d2816e221aa-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:04:50.253156 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.253134    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"]
Aug 04 00:04:50.257119 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.257098    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3113ce29-74c4f6d598-8plb8"]
Aug 04 00:04:50.267566 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.267528    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"]
Aug 04 00:04:50.272031 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.272010    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3113ce29-f9d5cf8f6-km9tk"]
Aug 04 00:04:50.282628 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.282605    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"]
Aug 04 00:04:50.286338 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.286318    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3113ce29-6d45689bc4-cmhgq"]
Aug 04 00:04:50.765867 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.765784    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a5da00d-6661-4e90-8cb9-82d9f659676f" path="/var/lib/kubelet/pods/0a5da00d-6661-4e90-8cb9-82d9f659676f/volumes"
Aug 04 00:04:50.766342 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.766323    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc1120e3-8963-40dd-bfb4-7d2816e221aa" path="/var/lib/kubelet/pods/dc1120e3-8963-40dd-bfb4-7d2816e221aa/volumes"
Aug 04 00:04:50.766775 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:50.766758    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e57593fb-3313-4c89-982e-bdc125437a71" path="/var/lib/kubelet/pods/e57593fb-3313-4c89-982e-bdc125437a71/volumes"
Aug 04 00:04:55.225182 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.225151    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"]
Aug 04 00:04:55.225546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.225503    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a5da00d-6661-4e90-8cb9-82d9f659676f" containerName="payload-processing"
Aug 04 00:04:55.225546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.225513    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a5da00d-6661-4e90-8cb9-82d9f659676f" containerName="payload-processing"
Aug 04 00:04:55.225546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.225540    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc1120e3-8963-40dd-bfb4-7d2816e221aa" containerName="maas-api"
Aug 04 00:04:55.225546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.225546    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc1120e3-8963-40dd-bfb4-7d2816e221aa" containerName="maas-api"
Aug 04 00:04:55.225669 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.225555    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e57593fb-3313-4c89-982e-bdc125437a71" containerName="payload-pre-processing"
Aug 04 00:04:55.225669 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.225560    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e57593fb-3313-4c89-982e-bdc125437a71" containerName="payload-pre-processing"
Aug 04 00:04:55.225669 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.225614    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="e57593fb-3313-4c89-982e-bdc125437a71" containerName="payload-pre-processing"
Aug 04 00:04:55.225669 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.225623    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a5da00d-6661-4e90-8cb9-82d9f659676f" containerName="payload-processing"
Aug 04 00:04:55.225669 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.225630    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc1120e3-8963-40dd-bfb4-7d2816e221aa" containerName="maas-api"
Aug 04 00:04:55.230410 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.230391    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:55.232759 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.232738    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7b45e0cc-serving-cert\""
Aug 04 00:04:55.240560 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.240531    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"]
Aug 04 00:04:55.383836 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.383804    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-openshift-service-ca\") pod \"maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:55.383970 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.383855    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgrwb\" (UniqueName: \"kubernetes.io/projected/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-kube-api-access-xgrwb\") pod \"maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:55.383970 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.383927    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-maas-api-tls\") pod \"maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:55.485318 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.485258    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-maas-api-tls\") pod \"maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:55.485420 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.485326    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-openshift-service-ca\") pod \"maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:55.485469 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.485420    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgrwb\" (UniqueName: \"kubernetes.io/projected/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-kube-api-access-xgrwb\") pod \"maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:55.486032 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.485974    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-openshift-service-ca\") pod \"maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:55.487659 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.487634    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-maas-api-tls\") pod \"maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:55.496629 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.496606    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgrwb\" (UniqueName: \"kubernetes.io/projected/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-kube-api-access-xgrwb\") pod \"maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:55.542682 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.542662    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:55.672204 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.672165    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"]
Aug 04 00:04:55.674182 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:04:55.674138    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc659d9c_2e4b_48c4_b81c_b7bb45ed9082.slice/crio-f664e3d69c33c285201229a3f5ff7e505254608b9469ca4210a76f600190a55c WatchSource:0}: Error finding container f664e3d69c33c285201229a3f5ff7e505254608b9469ca4210a76f600190a55c: Status 404 returned error can't find the container with id f664e3d69c33c285201229a3f5ff7e505254608b9469ca4210a76f600190a55c
Aug 04 00:04:55.871380 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.871306    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"]
Aug 04 00:04:55.875114 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.875099    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:04:55.877271 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.877255    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-7b45e0cc\""
Aug 04 00:04:55.884062 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.884035    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"]
Aug 04 00:04:55.910670 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.910640    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"]
Aug 04 00:04:55.913875 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.913853    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:04:55.916060 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.916040    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-7b45e0cc-dockercfg-v79jd\""
Aug 04 00:04:55.923140 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.923121    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"]
Aug 04 00:04:55.961934 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.961912    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj" event={"ID":"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082","Type":"ContainerStarted","Data":"f664e3d69c33c285201229a3f5ff7e505254608b9469ca4210a76f600190a55c"}
Aug 04 00:04:55.990687 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.990661    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nr9r\" (UniqueName: \"kubernetes.io/projected/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-kube-api-access-6nr9r\") pod \"payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx\" (UID: \"4babcfaa-a6e9-48c3-bdf8-61083a6582fd\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:04:55.990794 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:55.990699    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx\" (UID: \"4babcfaa-a6e9-48c3-bdf8-61083a6582fd\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:04:56.091810 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.091782    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1b30792-729d-42f6-9252-ef10e21ea28f-plugins-config-volume\") pod \"payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f\" (UID: \"c1b30792-729d-42f6-9252-ef10e21ea28f\") " pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:04:56.091949 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.091832    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp87h\" (UniqueName: \"kubernetes.io/projected/c1b30792-729d-42f6-9252-ef10e21ea28f-kube-api-access-fp87h\") pod \"payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f\" (UID: \"c1b30792-729d-42f6-9252-ef10e21ea28f\") " pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:04:56.091949 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.091870    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nr9r\" (UniqueName: \"kubernetes.io/projected/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-kube-api-access-6nr9r\") pod \"payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx\" (UID: \"4babcfaa-a6e9-48c3-bdf8-61083a6582fd\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:04:56.091949 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.091894    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx\" (UID: \"4babcfaa-a6e9-48c3-bdf8-61083a6582fd\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:04:56.092619 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.092597    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx\" (UID: \"4babcfaa-a6e9-48c3-bdf8-61083a6582fd\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:04:56.101553 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.101531    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nr9r\" (UniqueName: \"kubernetes.io/projected/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-kube-api-access-6nr9r\") pod \"payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx\" (UID: \"4babcfaa-a6e9-48c3-bdf8-61083a6582fd\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:04:56.187065 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.187040    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:04:56.192483 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.192448    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1b30792-729d-42f6-9252-ef10e21ea28f-plugins-config-volume\") pod \"payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f\" (UID: \"c1b30792-729d-42f6-9252-ef10e21ea28f\") " pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:04:56.192595 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.192514    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp87h\" (UniqueName: \"kubernetes.io/projected/c1b30792-729d-42f6-9252-ef10e21ea28f-kube-api-access-fp87h\") pod \"payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f\" (UID: \"c1b30792-729d-42f6-9252-ef10e21ea28f\") " pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:04:56.193163 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.193135    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1b30792-729d-42f6-9252-ef10e21ea28f-plugins-config-volume\") pod \"payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f\" (UID: \"c1b30792-729d-42f6-9252-ef10e21ea28f\") " pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:04:56.201973 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.201952    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp87h\" (UniqueName: \"kubernetes.io/projected/c1b30792-729d-42f6-9252-ef10e21ea28f-kube-api-access-fp87h\") pod \"payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f\" (UID: \"c1b30792-729d-42f6-9252-ef10e21ea28f\") " pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:04:56.223874 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.223844    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:04:56.335235 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.335203    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"]
Aug 04 00:04:56.337102 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:04:56.337065    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4babcfaa_a6e9_48c3_bdf8_61083a6582fd.slice/crio-a04b900404300e0968b6dbcbf410eedce90e469e800a21b60dc6b4e30ba96b26 WatchSource:0}: Error finding container a04b900404300e0968b6dbcbf410eedce90e469e800a21b60dc6b4e30ba96b26: Status 404 returned error can't find the container with id a04b900404300e0968b6dbcbf410eedce90e469e800a21b60dc6b4e30ba96b26
Aug 04 00:04:56.374486 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.374465    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"]
Aug 04 00:04:56.376240 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:04:56.376211    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b30792_729d_42f6_9252_ef10e21ea28f.slice/crio-821b1075ba7d76cbf1f2e75ff9025bfc27f6bc9124a5676f9bb97bf2124c81df WatchSource:0}: Error finding container 821b1075ba7d76cbf1f2e75ff9025bfc27f6bc9124a5676f9bb97bf2124c81df: Status 404 returned error can't find the container with id 821b1075ba7d76cbf1f2e75ff9025bfc27f6bc9124a5676f9bb97bf2124c81df
Aug 04 00:04:56.968527 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.968489    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj" event={"ID":"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082","Type":"ContainerStarted","Data":"38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1"}
Aug 04 00:04:56.968738 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.968611    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:04:56.970089 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.970070    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f" event={"ID":"c1b30792-729d-42f6-9252-ef10e21ea28f","Type":"ContainerStarted","Data":"a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc"}
Aug 04 00:04:56.970171 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.970105    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f" event={"ID":"c1b30792-729d-42f6-9252-ef10e21ea28f","Type":"ContainerStarted","Data":"821b1075ba7d76cbf1f2e75ff9025bfc27f6bc9124a5676f9bb97bf2124c81df"}
Aug 04 00:04:56.970235 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.970167    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:04:56.971572 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.971547    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx" event={"ID":"4babcfaa-a6e9-48c3-bdf8-61083a6582fd","Type":"ContainerStarted","Data":"bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025"}
Aug 04 00:04:56.971766 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.971589    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx" event={"ID":"4babcfaa-a6e9-48c3-bdf8-61083a6582fd","Type":"ContainerStarted","Data":"a04b900404300e0968b6dbcbf410eedce90e469e800a21b60dc6b4e30ba96b26"}
Aug 04 00:04:56.971766 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.971616    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:04:56.987077 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:56.987034    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj" podStartSLOduration=1.7096615499999999 podStartE2EDuration="1.987021472s" totalImagesPullingTime="277.359922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:04:56.985510831 +0000 UTC m=+2104.687217494" watchObservedRunningTime="2026-08-04 00:04:56.987021472 +0000 UTC m=+2104.688728135"
Aug 04 00:04:57.001042 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:57.000996    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f" podStartSLOduration=2.000985675 podStartE2EDuration="2.000985675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:04:56.999926984 +0000 UTC m=+2104.701633646" watchObservedRunningTime="2026-08-04 00:04:57.000985675 +0000 UTC m=+2104.702692338"
Aug 04 00:04:57.014323 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:04:57.014272    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx" podStartSLOduration=2.014256719 podStartE2EDuration="2.014256719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:04:57.013903869 +0000 UTC m=+2104.715610534" watchObservedRunningTime="2026-08-04 00:04:57.014256719 +0000 UTC m=+2104.715963382"
Aug 04 00:05:02.981376 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:02.981346    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:05:07.978440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:07.978412    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:05:07.978887 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:07.978477    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:05:35.780246 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:35.780215    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"]
Aug 04 00:05:35.780888 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:35.780429    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj" podUID="bc659d9c-2e4b-48c4-b81c-b7bb45ed9082" containerName="maas-api" containerID="cri-o://38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1" gracePeriod=30
Aug 04 00:05:35.826740 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:35.826697    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"]
Aug 04 00:05:35.826948 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:35.826924    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f" podUID="c1b30792-729d-42f6-9252-ef10e21ea28f" containerName="payload-processing" containerID="cri-o://a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc" gracePeriod=30
Aug 04 00:05:35.831810 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:35.831734    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"]
Aug 04 00:05:35.832320 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:35.832233    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx" podUID="4babcfaa-a6e9-48c3-bdf8-61083a6582fd" containerName="payload-pre-processing" containerID="cri-o://bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025" gracePeriod=30
Aug 04 00:05:36.118212 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.118189    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:05:36.120216 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.120193    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:05:36.129433 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.129414    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:05:36.155968 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.155937    2553 generic.go:356] "Generic (PLEG): container finished" podID="4babcfaa-a6e9-48c3-bdf8-61083a6582fd" containerID="bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025" exitCode=0
Aug 04 00:05:36.156129 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.155980    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"
Aug 04 00:05:36.156129 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.156029    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx" event={"ID":"4babcfaa-a6e9-48c3-bdf8-61083a6582fd","Type":"ContainerDied","Data":"bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025"}
Aug 04 00:05:36.156129 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.156075    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx" event={"ID":"4babcfaa-a6e9-48c3-bdf8-61083a6582fd","Type":"ContainerDied","Data":"a04b900404300e0968b6dbcbf410eedce90e469e800a21b60dc6b4e30ba96b26"}
Aug 04 00:05:36.156129 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.156103    2553 scope.go:117] "RemoveContainer" containerID="bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025"
Aug 04 00:05:36.157539 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.157512    2553 generic.go:356] "Generic (PLEG): container finished" podID="bc659d9c-2e4b-48c4-b81c-b7bb45ed9082" containerID="38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1" exitCode=0
Aug 04 00:05:36.157539 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.157543    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj" event={"ID":"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082","Type":"ContainerDied","Data":"38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1"}
Aug 04 00:05:36.157539 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.157575    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"
Aug 04 00:05:36.157938 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.157584    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj" event={"ID":"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082","Type":"ContainerDied","Data":"f664e3d69c33c285201229a3f5ff7e505254608b9469ca4210a76f600190a55c"}
Aug 04 00:05:36.158799 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.158777    2553 generic.go:356] "Generic (PLEG): container finished" podID="c1b30792-729d-42f6-9252-ef10e21ea28f" containerID="a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc" exitCode=0
Aug 04 00:05:36.158911 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.158807    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f" event={"ID":"c1b30792-729d-42f6-9252-ef10e21ea28f","Type":"ContainerDied","Data":"a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc"}
Aug 04 00:05:36.158911 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.158817    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"
Aug 04 00:05:36.158911 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.158842    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f" event={"ID":"c1b30792-729d-42f6-9252-ef10e21ea28f","Type":"ContainerDied","Data":"821b1075ba7d76cbf1f2e75ff9025bfc27f6bc9124a5676f9bb97bf2124c81df"}
Aug 04 00:05:36.165941 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.165890    2553 scope.go:117] "RemoveContainer" containerID="bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025"
Aug 04 00:05:36.166314 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:05:36.166214    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025\": container with ID starting with bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025 not found: ID does not exist" containerID="bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025"
Aug 04 00:05:36.166429 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.166354    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025"} err="failed to get container status \"bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025\": rpc error: code = NotFound desc = could not find container \"bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025\": container with ID starting with bc1e9df03dbaae9cc873b4ee44944d62f2ca00c8fc9cf05753b99f132254a025 not found: ID does not exist"
Aug 04 00:05:36.166429 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.166384    2553 scope.go:117] "RemoveContainer" containerID="38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1"
Aug 04 00:05:36.175823 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.175805    2553 scope.go:117] "RemoveContainer" containerID="38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1"
Aug 04 00:05:36.176077 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:05:36.176059    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1\": container with ID starting with 38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1 not found: ID does not exist" containerID="38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1"
Aug 04 00:05:36.176162 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.176087    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1"} err="failed to get container status \"38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1\": rpc error: code = NotFound desc = could not find container \"38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1\": container with ID starting with 38d330a8578f1dc2f90674a289533ab009829d26b04ce5b60330d46a25f55ab1 not found: ID does not exist"
Aug 04 00:05:36.176162 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.176109    2553 scope.go:117] "RemoveContainer" containerID="a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc"
Aug 04 00:05:36.185088 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.185070    2553 scope.go:117] "RemoveContainer" containerID="a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc"
Aug 04 00:05:36.185347 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:05:36.185329    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc\": container with ID starting with a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc not found: ID does not exist" containerID="a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc"
Aug 04 00:05:36.185409 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.185351    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc"} err="failed to get container status \"a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc\": rpc error: code = NotFound desc = could not find container \"a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc\": container with ID starting with a134663cf1ce30749320d9b31c61fcaebba49914569a3f6b8f707fdb5d271ddc not found: ID does not exist"
Aug 04 00:05:36.247670 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.247644    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nr9r\" (UniqueName: \"kubernetes.io/projected/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-kube-api-access-6nr9r\") pod \"4babcfaa-a6e9-48c3-bdf8-61083a6582fd\" (UID: \"4babcfaa-a6e9-48c3-bdf8-61083a6582fd\") "
Aug 04 00:05:36.247824 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.247681    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1b30792-729d-42f6-9252-ef10e21ea28f-plugins-config-volume\") pod \"c1b30792-729d-42f6-9252-ef10e21ea28f\" (UID: \"c1b30792-729d-42f6-9252-ef10e21ea28f\") "
Aug 04 00:05:36.247824 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.247734    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp87h\" (UniqueName: \"kubernetes.io/projected/c1b30792-729d-42f6-9252-ef10e21ea28f-kube-api-access-fp87h\") pod \"c1b30792-729d-42f6-9252-ef10e21ea28f\" (UID: \"c1b30792-729d-42f6-9252-ef10e21ea28f\") "
Aug 04 00:05:36.247824 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.247769    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-plugins-config-volume\") pod \"4babcfaa-a6e9-48c3-bdf8-61083a6582fd\" (UID: \"4babcfaa-a6e9-48c3-bdf8-61083a6582fd\") "
Aug 04 00:05:36.247988 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.247838    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-maas-api-tls\") pod \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") "
Aug 04 00:05:36.247988 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.247913    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-openshift-service-ca\") pod \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") "
Aug 04 00:05:36.247988 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.247949    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgrwb\" (UniqueName: \"kubernetes.io/projected/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-kube-api-access-xgrwb\") pod \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\" (UID: \"bc659d9c-2e4b-48c4-b81c-b7bb45ed9082\") "
Aug 04 00:05:36.248307 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.248278    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4babcfaa-a6e9-48c3-bdf8-61083a6582fd" (UID: "4babcfaa-a6e9-48c3-bdf8-61083a6582fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:05:36.248524 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.248300    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bc659d9c-2e4b-48c4-b81c-b7bb45ed9082" (UID: "bc659d9c-2e4b-48c4-b81c-b7bb45ed9082"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:05:36.248524 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.248359    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1b30792-729d-42f6-9252-ef10e21ea28f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c1b30792-729d-42f6-9252-ef10e21ea28f" (UID: "c1b30792-729d-42f6-9252-ef10e21ea28f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:05:36.250048 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.250013    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1b30792-729d-42f6-9252-ef10e21ea28f-kube-api-access-fp87h" (OuterVolumeSpecName: "kube-api-access-fp87h") pod "c1b30792-729d-42f6-9252-ef10e21ea28f" (UID: "c1b30792-729d-42f6-9252-ef10e21ea28f"). InnerVolumeSpecName "kube-api-access-fp87h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:05:36.250135 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.250048    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-kube-api-access-6nr9r" (OuterVolumeSpecName: "kube-api-access-6nr9r") pod "4babcfaa-a6e9-48c3-bdf8-61083a6582fd" (UID: "4babcfaa-a6e9-48c3-bdf8-61083a6582fd"). InnerVolumeSpecName "kube-api-access-6nr9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:05:36.250135 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.250059    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bc659d9c-2e4b-48c4-b81c-b7bb45ed9082" (UID: "bc659d9c-2e4b-48c4-b81c-b7bb45ed9082"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:05:36.250288 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.250272    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-kube-api-access-xgrwb" (OuterVolumeSpecName: "kube-api-access-xgrwb") pod "bc659d9c-2e4b-48c4-b81c-b7bb45ed9082" (UID: "bc659d9c-2e4b-48c4-b81c-b7bb45ed9082"). InnerVolumeSpecName "kube-api-access-xgrwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:05:36.349467 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.349396    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:05:36.349467 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.349423    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:05:36.349467 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.349433    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:05:36.349467 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.349445    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgrwb\" (UniqueName: \"kubernetes.io/projected/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082-kube-api-access-xgrwb\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:05:36.349467 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.349453    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nr9r\" (UniqueName: \"kubernetes.io/projected/4babcfaa-a6e9-48c3-bdf8-61083a6582fd-kube-api-access-6nr9r\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:05:36.349467 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.349462    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1b30792-729d-42f6-9252-ef10e21ea28f-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:05:36.349467 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.349471    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp87h\" (UniqueName: \"kubernetes.io/projected/c1b30792-729d-42f6-9252-ef10e21ea28f-kube-api-access-fp87h\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:05:36.482211 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.482182    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"]
Aug 04 00:05:36.488047 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.487989    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7b45e0cc-76bdc8867b-jgpfx"]
Aug 04 00:05:36.497691 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.497662    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"]
Aug 04 00:05:36.503645 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.503618    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7b45e0cc-58578b7858-hqb4f"]
Aug 04 00:05:36.513007 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.512971    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"]
Aug 04 00:05:36.514965 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.514946    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7b45e0cc-54df8c7d97-kjfvj"]
Aug 04 00:05:36.764744 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.764693    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4babcfaa-a6e9-48c3-bdf8-61083a6582fd" path="/var/lib/kubelet/pods/4babcfaa-a6e9-48c3-bdf8-61083a6582fd/volumes"
Aug 04 00:05:36.765192 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.765168    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc659d9c-2e4b-48c4-b81c-b7bb45ed9082" path="/var/lib/kubelet/pods/bc659d9c-2e4b-48c4-b81c-b7bb45ed9082/volumes"
Aug 04 00:05:36.765636 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:05:36.765617    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1b30792-729d-42f6-9252-ef10e21ea28f" path="/var/lib/kubelet/pods/c1b30792-729d-42f6-9252-ef10e21ea28f/volumes"
Aug 04 00:07:37.606962 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.606925    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"]
Aug 04 00:07:37.607546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.607481    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc659d9c-2e4b-48c4-b81c-b7bb45ed9082" containerName="maas-api"
Aug 04 00:07:37.607546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.607501    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc659d9c-2e4b-48c4-b81c-b7bb45ed9082" containerName="maas-api"
Aug 04 00:07:37.607546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.607529    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1b30792-729d-42f6-9252-ef10e21ea28f" containerName="payload-processing"
Aug 04 00:07:37.607546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.607538    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1b30792-729d-42f6-9252-ef10e21ea28f" containerName="payload-processing"
Aug 04 00:07:37.607801 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.607569    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4babcfaa-a6e9-48c3-bdf8-61083a6582fd" containerName="payload-pre-processing"
Aug 04 00:07:37.607801 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.607579    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4babcfaa-a6e9-48c3-bdf8-61083a6582fd" containerName="payload-pre-processing"
Aug 04 00:07:37.607801 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.607667    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="4babcfaa-a6e9-48c3-bdf8-61083a6582fd" containerName="payload-pre-processing"
Aug 04 00:07:37.607801 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.607681    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1b30792-729d-42f6-9252-ef10e21ea28f" containerName="payload-processing"
Aug 04 00:07:37.607801 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.607692    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc659d9c-2e4b-48c4-b81c-b7bb45ed9082" containerName="maas-api"
Aug 04 00:07:37.611011 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.610988    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:37.613443 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.613420    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2c79f557-serving-cert\""
Aug 04 00:07:37.619901 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.619878    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"]
Aug 04 00:07:37.759002 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.758971    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/962557ce-448e-4353-b671-18eadce63c70-maas-api-tls\") pod \"maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:37.759002 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.759001    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdd4p\" (UniqueName: \"kubernetes.io/projected/962557ce-448e-4353-b671-18eadce63c70-kube-api-access-wdd4p\") pod \"maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:37.759220 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.759055    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/962557ce-448e-4353-b671-18eadce63c70-openshift-service-ca\") pod \"maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:37.859867 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.859783    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/962557ce-448e-4353-b671-18eadce63c70-openshift-service-ca\") pod \"maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:37.860013 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.859890    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/962557ce-448e-4353-b671-18eadce63c70-maas-api-tls\") pod \"maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:37.860013 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.859915    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdd4p\" (UniqueName: \"kubernetes.io/projected/962557ce-448e-4353-b671-18eadce63c70-kube-api-access-wdd4p\") pod \"maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:37.860085 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:07:37.860029    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-serving-cert: secret "maas-api-e2e-mt-2c79f557-serving-cert" not found
Aug 04 00:07:37.860128 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:07:37.860094    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962557ce-448e-4353-b671-18eadce63c70-maas-api-tls podName:962557ce-448e-4353-b671-18eadce63c70 nodeName:}" failed. No retries permitted until 2026-08-04 00:07:38.360073699 +0000 UTC m=+2266.061780340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/962557ce-448e-4353-b671-18eadce63c70-maas-api-tls") pod "maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm" (UID: "962557ce-448e-4353-b671-18eadce63c70") : secret "maas-api-e2e-mt-2c79f557-serving-cert" not found
Aug 04 00:07:37.860555 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.860537    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/962557ce-448e-4353-b671-18eadce63c70-openshift-service-ca\") pod \"maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:37.869108 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:37.869081    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdd4p\" (UniqueName: \"kubernetes.io/projected/962557ce-448e-4353-b671-18eadce63c70-kube-api-access-wdd4p\") pod \"maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:38.232931 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.232894    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"]
Aug 04 00:07:38.236610 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.236595    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:07:38.239086 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.239064    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2c79f557-dockercfg-tdlw4\""
Aug 04 00:07:38.239207 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.239066    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2c79f557\""
Aug 04 00:07:38.246385 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.246363    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"]
Aug 04 00:07:38.270182 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.270155    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"]
Aug 04 00:07:38.274272 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.274251    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:07:38.281216 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.281191    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"]
Aug 04 00:07:38.365112 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.365079    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/962557ce-448e-4353-b671-18eadce63c70-maas-api-tls\") pod \"maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:38.365309 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.365120    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/373906b4-36a4-46f7-ad54-055dab5165bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4\" (UID: \"373906b4-36a4-46f7-ad54-055dab5165bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:07:38.365309 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.365147    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c516b750-0dc8-478b-aec4-9d55335dfb3d-plugins-config-volume\") pod \"payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4\" (UID: \"c516b750-0dc8-478b-aec4-9d55335dfb3d\") " pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:07:38.365309 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.365173    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kqxw\" (UniqueName: \"kubernetes.io/projected/373906b4-36a4-46f7-ad54-055dab5165bf-kube-api-access-4kqxw\") pod \"payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4\" (UID: \"373906b4-36a4-46f7-ad54-055dab5165bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:07:38.365309 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.365254    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zglsl\" (UniqueName: \"kubernetes.io/projected/c516b750-0dc8-478b-aec4-9d55335dfb3d-kube-api-access-zglsl\") pod \"payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4\" (UID: \"c516b750-0dc8-478b-aec4-9d55335dfb3d\") " pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:07:38.368010 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.367977    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/962557ce-448e-4353-b671-18eadce63c70-maas-api-tls\") pod \"maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:38.466081 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.466048    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/373906b4-36a4-46f7-ad54-055dab5165bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4\" (UID: \"373906b4-36a4-46f7-ad54-055dab5165bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:07:38.466248 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.466092    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c516b750-0dc8-478b-aec4-9d55335dfb3d-plugins-config-volume\") pod \"payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4\" (UID: \"c516b750-0dc8-478b-aec4-9d55335dfb3d\") " pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:07:38.466248 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.466112    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kqxw\" (UniqueName: \"kubernetes.io/projected/373906b4-36a4-46f7-ad54-055dab5165bf-kube-api-access-4kqxw\") pod \"payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4\" (UID: \"373906b4-36a4-46f7-ad54-055dab5165bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:07:38.466248 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.466150    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zglsl\" (UniqueName: \"kubernetes.io/projected/c516b750-0dc8-478b-aec4-9d55335dfb3d-kube-api-access-zglsl\") pod \"payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4\" (UID: \"c516b750-0dc8-478b-aec4-9d55335dfb3d\") " pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:07:38.466805 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.466787    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c516b750-0dc8-478b-aec4-9d55335dfb3d-plugins-config-volume\") pod \"payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4\" (UID: \"c516b750-0dc8-478b-aec4-9d55335dfb3d\") " pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:07:38.466864 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.466807    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/373906b4-36a4-46f7-ad54-055dab5165bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4\" (UID: \"373906b4-36a4-46f7-ad54-055dab5165bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:07:38.475417 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.475396    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zglsl\" (UniqueName: \"kubernetes.io/projected/c516b750-0dc8-478b-aec4-9d55335dfb3d-kube-api-access-zglsl\") pod \"payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4\" (UID: \"c516b750-0dc8-478b-aec4-9d55335dfb3d\") " pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:07:38.475525 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.475450    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kqxw\" (UniqueName: \"kubernetes.io/projected/373906b4-36a4-46f7-ad54-055dab5165bf-kube-api-access-4kqxw\") pod \"payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4\" (UID: \"373906b4-36a4-46f7-ad54-055dab5165bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:07:38.523519 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.523438    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:38.548610 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.548578    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:07:38.586280 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.586249    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:07:38.689103 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.689068    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"]
Aug 04 00:07:38.692482 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:07:38.692449    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962557ce_448e_4353_b671_18eadce63c70.slice/crio-5d7effaaaff4f943bb67c7d15d702f15b26d7476571a215094c362b34eaab4f3 WatchSource:0}: Error finding container 5d7effaaaff4f943bb67c7d15d702f15b26d7476571a215094c362b34eaab4f3: Status 404 returned error can't find the container with id 5d7effaaaff4f943bb67c7d15d702f15b26d7476571a215094c362b34eaab4f3
Aug 04 00:07:38.711192 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.711151    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"]
Aug 04 00:07:38.713244 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:07:38.713217    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod373906b4_36a4_46f7_ad54_055dab5165bf.slice/crio-18a94e8a7d77b79bf627a0318bff7045b81972b56e04d595bae5d2ba8891cbd8 WatchSource:0}: Error finding container 18a94e8a7d77b79bf627a0318bff7045b81972b56e04d595bae5d2ba8891cbd8: Status 404 returned error can't find the container with id 18a94e8a7d77b79bf627a0318bff7045b81972b56e04d595bae5d2ba8891cbd8
Aug 04 00:07:38.754629 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.754605    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"]
Aug 04 00:07:38.755287 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.755255    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4" event={"ID":"373906b4-36a4-46f7-ad54-055dab5165bf","Type":"ContainerStarted","Data":"18a94e8a7d77b79bf627a0318bff7045b81972b56e04d595bae5d2ba8891cbd8"}
Aug 04 00:07:38.756649 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:38.756621    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm" event={"ID":"962557ce-448e-4353-b671-18eadce63c70","Type":"ContainerStarted","Data":"5d7effaaaff4f943bb67c7d15d702f15b26d7476571a215094c362b34eaab4f3"}
Aug 04 00:07:38.756943 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:07:38.756915    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc516b750_0dc8_478b_aec4_9d55335dfb3d.slice/crio-4481fce33f0de33f42123242e2c58d605a9fc177c2526478ab4264796824647e WatchSource:0}: Error finding container 4481fce33f0de33f42123242e2c58d605a9fc177c2526478ab4264796824647e: Status 404 returned error can't find the container with id 4481fce33f0de33f42123242e2c58d605a9fc177c2526478ab4264796824647e
Aug 04 00:07:39.764253 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:39.764217    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm" event={"ID":"962557ce-448e-4353-b671-18eadce63c70","Type":"ContainerStarted","Data":"054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2"}
Aug 04 00:07:39.764760 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:39.764286    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:39.765925 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:39.765903    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4" event={"ID":"c516b750-0dc8-478b-aec4-9d55335dfb3d","Type":"ContainerStarted","Data":"98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16"}
Aug 04 00:07:39.766092 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:39.766073    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4" event={"ID":"c516b750-0dc8-478b-aec4-9d55335dfb3d","Type":"ContainerStarted","Data":"4481fce33f0de33f42123242e2c58d605a9fc177c2526478ab4264796824647e"}
Aug 04 00:07:39.766211 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:39.766107    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:07:39.767402 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:39.767382    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4" event={"ID":"373906b4-36a4-46f7-ad54-055dab5165bf","Type":"ContainerStarted","Data":"55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b"}
Aug 04 00:07:39.767529 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:39.767513    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:07:39.780094 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:39.780052    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm" podStartSLOduration=2.169416501 podStartE2EDuration="2.780036796s" totalImagesPullingTime="610.620295ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:07:39.779256432 +0000 UTC m=+2267.480963096" watchObservedRunningTime="2026-08-04 00:07:39.780036796 +0000 UTC m=+2267.481743460"
Aug 04 00:07:39.797177 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:39.797132    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4" podStartSLOduration=1.797116834 podStartE2EDuration="1.797116834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:07:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:07:39.795784361 +0000 UTC m=+2267.497491037" watchObservedRunningTime="2026-08-04 00:07:39.797116834 +0000 UTC m=+2267.498823563"
Aug 04 00:07:39.814173 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:39.814130    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4" podStartSLOduration=1.814113621 podStartE2EDuration="1.814113621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:07:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:07:39.812265927 +0000 UTC m=+2267.513972591" watchObservedRunningTime="2026-08-04 00:07:39.814113621 +0000 UTC m=+2267.515820287"
Aug 04 00:07:45.777342 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:45.777307    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:07:50.774533 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:50.774506    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:07:50.774949 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:07:50.774649    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:08:19.422133 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.422089    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"]
Aug 04 00:08:19.422699 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.422356    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm" podUID="962557ce-448e-4353-b671-18eadce63c70" containerName="maas-api" containerID="cri-o://054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2" gracePeriod=30
Aug 04 00:08:19.525595 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.525558    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"]
Aug 04 00:08:19.525839 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.525814    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4" podUID="c516b750-0dc8-478b-aec4-9d55335dfb3d" containerName="payload-processing" containerID="cri-o://98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16" gracePeriod=30
Aug 04 00:08:19.533148 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.533118    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"]
Aug 04 00:08:19.533387 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.533361    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4" podUID="373906b4-36a4-46f7-ad54-055dab5165bf" containerName="payload-pre-processing" containerID="cri-o://55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b" gracePeriod=30
Aug 04 00:08:19.697268 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.697241    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:08:19.777064 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.777034    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:08:19.815765 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.815742    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:08:19.862408 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.862381    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdd4p\" (UniqueName: \"kubernetes.io/projected/962557ce-448e-4353-b671-18eadce63c70-kube-api-access-wdd4p\") pod \"962557ce-448e-4353-b671-18eadce63c70\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") "
Aug 04 00:08:19.862590 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.862445    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zglsl\" (UniqueName: \"kubernetes.io/projected/c516b750-0dc8-478b-aec4-9d55335dfb3d-kube-api-access-zglsl\") pod \"c516b750-0dc8-478b-aec4-9d55335dfb3d\" (UID: \"c516b750-0dc8-478b-aec4-9d55335dfb3d\") "
Aug 04 00:08:19.862590 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.862523    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/962557ce-448e-4353-b671-18eadce63c70-maas-api-tls\") pod \"962557ce-448e-4353-b671-18eadce63c70\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") "
Aug 04 00:08:19.862590 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.862555    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/962557ce-448e-4353-b671-18eadce63c70-openshift-service-ca\") pod \"962557ce-448e-4353-b671-18eadce63c70\" (UID: \"962557ce-448e-4353-b671-18eadce63c70\") "
Aug 04 00:08:19.862590 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.862572    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c516b750-0dc8-478b-aec4-9d55335dfb3d-plugins-config-volume\") pod \"c516b750-0dc8-478b-aec4-9d55335dfb3d\" (UID: \"c516b750-0dc8-478b-aec4-9d55335dfb3d\") "
Aug 04 00:08:19.862981 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.862950    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/962557ce-448e-4353-b671-18eadce63c70-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "962557ce-448e-4353-b671-18eadce63c70" (UID: "962557ce-448e-4353-b671-18eadce63c70"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:08:19.863074 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.863030    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c516b750-0dc8-478b-aec4-9d55335dfb3d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c516b750-0dc8-478b-aec4-9d55335dfb3d" (UID: "c516b750-0dc8-478b-aec4-9d55335dfb3d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:08:19.863465 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.863438    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/962557ce-448e-4353-b671-18eadce63c70-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:08:19.863560 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.863466    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c516b750-0dc8-478b-aec4-9d55335dfb3d-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:08:19.864883 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.864859    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/962557ce-448e-4353-b671-18eadce63c70-kube-api-access-wdd4p" (OuterVolumeSpecName: "kube-api-access-wdd4p") pod "962557ce-448e-4353-b671-18eadce63c70" (UID: "962557ce-448e-4353-b671-18eadce63c70"). InnerVolumeSpecName "kube-api-access-wdd4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:08:19.864963 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.864881    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/962557ce-448e-4353-b671-18eadce63c70-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "962557ce-448e-4353-b671-18eadce63c70" (UID: "962557ce-448e-4353-b671-18eadce63c70"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:08:19.864963 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.864883    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c516b750-0dc8-478b-aec4-9d55335dfb3d-kube-api-access-zglsl" (OuterVolumeSpecName: "kube-api-access-zglsl") pod "c516b750-0dc8-478b-aec4-9d55335dfb3d" (UID: "c516b750-0dc8-478b-aec4-9d55335dfb3d"). InnerVolumeSpecName "kube-api-access-zglsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:08:19.964445 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.964412    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/373906b4-36a4-46f7-ad54-055dab5165bf-plugins-config-volume\") pod \"373906b4-36a4-46f7-ad54-055dab5165bf\" (UID: \"373906b4-36a4-46f7-ad54-055dab5165bf\") "
Aug 04 00:08:19.964653 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.964477    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kqxw\" (UniqueName: \"kubernetes.io/projected/373906b4-36a4-46f7-ad54-055dab5165bf-kube-api-access-4kqxw\") pod \"373906b4-36a4-46f7-ad54-055dab5165bf\" (UID: \"373906b4-36a4-46f7-ad54-055dab5165bf\") "
Aug 04 00:08:19.964792 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.964775    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/962557ce-448e-4353-b671-18eadce63c70-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:08:19.964864 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.964798    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdd4p\" (UniqueName: \"kubernetes.io/projected/962557ce-448e-4353-b671-18eadce63c70-kube-api-access-wdd4p\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:08:19.964864 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.964814    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zglsl\" (UniqueName: \"kubernetes.io/projected/c516b750-0dc8-478b-aec4-9d55335dfb3d-kube-api-access-zglsl\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:08:19.964937 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.964884    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/373906b4-36a4-46f7-ad54-055dab5165bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "373906b4-36a4-46f7-ad54-055dab5165bf" (UID: "373906b4-36a4-46f7-ad54-055dab5165bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:08:19.966685 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.966666    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/373906b4-36a4-46f7-ad54-055dab5165bf-kube-api-access-4kqxw" (OuterVolumeSpecName: "kube-api-access-4kqxw") pod "373906b4-36a4-46f7-ad54-055dab5165bf" (UID: "373906b4-36a4-46f7-ad54-055dab5165bf"). InnerVolumeSpecName "kube-api-access-4kqxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:08:19.971877 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.971853    2553 generic.go:356] "Generic (PLEG): container finished" podID="373906b4-36a4-46f7-ad54-055dab5165bf" containerID="55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b" exitCode=0
Aug 04 00:08:19.971984 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.971960    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"
Aug 04 00:08:19.971984 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.971963    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4" event={"ID":"373906b4-36a4-46f7-ad54-055dab5165bf","Type":"ContainerDied","Data":"55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b"}
Aug 04 00:08:19.972077 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.972019    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4" event={"ID":"373906b4-36a4-46f7-ad54-055dab5165bf","Type":"ContainerDied","Data":"18a94e8a7d77b79bf627a0318bff7045b81972b56e04d595bae5d2ba8891cbd8"}
Aug 04 00:08:19.972077 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.972050    2553 scope.go:117] "RemoveContainer" containerID="55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b"
Aug 04 00:08:19.973222 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.973202    2553 generic.go:356] "Generic (PLEG): container finished" podID="962557ce-448e-4353-b671-18eadce63c70" containerID="054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2" exitCode=0
Aug 04 00:08:19.973332 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.973255    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"
Aug 04 00:08:19.973332 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.973289    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm" event={"ID":"962557ce-448e-4353-b671-18eadce63c70","Type":"ContainerDied","Data":"054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2"}
Aug 04 00:08:19.973431 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.973343    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm" event={"ID":"962557ce-448e-4353-b671-18eadce63c70","Type":"ContainerDied","Data":"5d7effaaaff4f943bb67c7d15d702f15b26d7476571a215094c362b34eaab4f3"}
Aug 04 00:08:19.974608 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.974586    2553 generic.go:356] "Generic (PLEG): container finished" podID="c516b750-0dc8-478b-aec4-9d55335dfb3d" containerID="98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16" exitCode=0
Aug 04 00:08:19.974725 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.974616    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"
Aug 04 00:08:19.974725 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.974624    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4" event={"ID":"c516b750-0dc8-478b-aec4-9d55335dfb3d","Type":"ContainerDied","Data":"98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16"}
Aug 04 00:08:19.974725 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.974660    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4" event={"ID":"c516b750-0dc8-478b-aec4-9d55335dfb3d","Type":"ContainerDied","Data":"4481fce33f0de33f42123242e2c58d605a9fc177c2526478ab4264796824647e"}
Aug 04 00:08:19.982361 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.982339    2553 scope.go:117] "RemoveContainer" containerID="55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b"
Aug 04 00:08:19.982664 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:08:19.982644    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b\": container with ID starting with 55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b not found: ID does not exist" containerID="55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b"
Aug 04 00:08:19.982750 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.982671    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b"} err="failed to get container status \"55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b\": rpc error: code = NotFound desc = could not find container \"55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b\": container with ID starting with 55b653ef43aee68d6f047c9d2ca73b7d8cfcd517a53c8fdefd8951637a8d044b not found: ID does not exist"
Aug 04 00:08:19.982750 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.982689    2553 scope.go:117] "RemoveContainer" containerID="054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2"
Aug 04 00:08:19.992273 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.992256    2553 scope.go:117] "RemoveContainer" containerID="054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2"
Aug 04 00:08:19.992559 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:08:19.992540    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2\": container with ID starting with 054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2 not found: ID does not exist" containerID="054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2"
Aug 04 00:08:19.992598 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.992566    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2"} err="failed to get container status \"054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2\": rpc error: code = NotFound desc = could not find container \"054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2\": container with ID starting with 054f4e251014c1ff10f6982f79207ac4f68e2d91ab75d90e66bfc97a6d652ba2 not found: ID does not exist"
Aug 04 00:08:19.992598 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:19.992581    2553 scope.go:117] "RemoveContainer" containerID="98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16"
Aug 04 00:08:20.001310 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.001282    2553 scope.go:117] "RemoveContainer" containerID="98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16"
Aug 04 00:08:20.001851 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.001668    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"]
Aug 04 00:08:20.001851 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:08:20.001665    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16\": container with ID starting with 98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16 not found: ID does not exist" containerID="98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16"
Aug 04 00:08:20.001851 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.001745    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16"} err="failed to get container status \"98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16\": rpc error: code = NotFound desc = could not find container \"98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16\": container with ID starting with 98a8c316171064265670e3a2947070523ce2a5cad1e5bae1dedabcf5e2875d16 not found: ID does not exist"
Aug 04 00:08:20.003837 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.003817    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c79f557-6986878cbd-2bnnm"]
Aug 04 00:08:20.015526 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.015502    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"]
Aug 04 00:08:20.019498 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.019477    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2c79f557-8dc989956-np9g4"]
Aug 04 00:08:20.030050 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.030022    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"]
Aug 04 00:08:20.035979 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.035951    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2c79f557-5587d67fd6-2hwc4"]
Aug 04 00:08:20.065356 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.065327    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/373906b4-36a4-46f7-ad54-055dab5165bf-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:08:20.065356 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.065354    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kqxw\" (UniqueName: \"kubernetes.io/projected/373906b4-36a4-46f7-ad54-055dab5165bf-kube-api-access-4kqxw\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:08:20.764646 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.764567    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="373906b4-36a4-46f7-ad54-055dab5165bf" path="/var/lib/kubelet/pods/373906b4-36a4-46f7-ad54-055dab5165bf/volumes"
Aug 04 00:08:20.765028 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.764988    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="962557ce-448e-4353-b671-18eadce63c70" path="/var/lib/kubelet/pods/962557ce-448e-4353-b671-18eadce63c70/volumes"
Aug 04 00:08:20.765324 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:08:20.765309    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c516b750-0dc8-478b-aec4-9d55335dfb3d" path="/var/lib/kubelet/pods/c516b750-0dc8-478b-aec4-9d55335dfb3d/volumes"
Aug 04 00:10:45.511156 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.511121    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"]
Aug 04 00:10:45.511559 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.511516    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c516b750-0dc8-478b-aec4-9d55335dfb3d" containerName="payload-processing"
Aug 04 00:10:45.511559 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.511528    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c516b750-0dc8-478b-aec4-9d55335dfb3d" containerName="payload-processing"
Aug 04 00:10:45.511559 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.511537    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="962557ce-448e-4353-b671-18eadce63c70" containerName="maas-api"
Aug 04 00:10:45.511559 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.511542    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="962557ce-448e-4353-b671-18eadce63c70" containerName="maas-api"
Aug 04 00:10:45.511693 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.511570    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="373906b4-36a4-46f7-ad54-055dab5165bf" containerName="payload-pre-processing"
Aug 04 00:10:45.511693 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.511578    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="373906b4-36a4-46f7-ad54-055dab5165bf" containerName="payload-pre-processing"
Aug 04 00:10:45.511693 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.511639    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="373906b4-36a4-46f7-ad54-055dab5165bf" containerName="payload-pre-processing"
Aug 04 00:10:45.511693 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.511651    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c516b750-0dc8-478b-aec4-9d55335dfb3d" containerName="payload-processing"
Aug 04 00:10:45.511693 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.511661    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="962557ce-448e-4353-b671-18eadce63c70" containerName="maas-api"
Aug 04 00:10:45.514930 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.514913    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:45.517224 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.517199    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-335654eb-serving-cert\""
Aug 04 00:10:45.526563 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.526538    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"]
Aug 04 00:10:45.605998 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.605966    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c17755e5-5958-446a-8c29-78b736690d0b-maas-api-tls\") pod \"maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:45.606170 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.606005    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfvg7\" (UniqueName: \"kubernetes.io/projected/c17755e5-5958-446a-8c29-78b736690d0b-kube-api-access-cfvg7\") pod \"maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:45.606170 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.606024    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c17755e5-5958-446a-8c29-78b736690d0b-openshift-service-ca\") pod \"maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:45.707254 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.707215    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c17755e5-5958-446a-8c29-78b736690d0b-maas-api-tls\") pod \"maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:45.707254 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.707257    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvg7\" (UniqueName: \"kubernetes.io/projected/c17755e5-5958-446a-8c29-78b736690d0b-kube-api-access-cfvg7\") pod \"maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:45.707456 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.707285    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c17755e5-5958-446a-8c29-78b736690d0b-openshift-service-ca\") pod \"maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:45.707456 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:10:45.707377    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-serving-cert: secret "maas-api-e2e-mt-335654eb-serving-cert" not found
Aug 04 00:10:45.707456 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:10:45.707444    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c17755e5-5958-446a-8c29-78b736690d0b-maas-api-tls podName:c17755e5-5958-446a-8c29-78b736690d0b nodeName:}" failed. No retries permitted until 2026-08-04 00:10:46.207424358 +0000 UTC m=+2453.909131000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c17755e5-5958-446a-8c29-78b736690d0b-maas-api-tls") pod "maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx" (UID: "c17755e5-5958-446a-8c29-78b736690d0b") : secret "maas-api-e2e-mt-335654eb-serving-cert" not found
Aug 04 00:10:45.707998 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.707980    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c17755e5-5958-446a-8c29-78b736690d0b-openshift-service-ca\") pod \"maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:45.719265 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:45.719235    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfvg7\" (UniqueName: \"kubernetes.io/projected/c17755e5-5958-446a-8c29-78b736690d0b-kube-api-access-cfvg7\") pod \"maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:46.211661 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.211628    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c17755e5-5958-446a-8c29-78b736690d0b-maas-api-tls\") pod \"maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:46.212371 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.212339    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"]
Aug 04 00:10:46.214321 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.214294    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c17755e5-5958-446a-8c29-78b736690d0b-maas-api-tls\") pod \"maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:46.215939 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.215922    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:10:46.218486 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.218467    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-335654eb\""
Aug 04 00:10:46.218622 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.218603    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-335654eb-dockercfg-hzkcb\""
Aug 04 00:10:46.224419 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.224394    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"]
Aug 04 00:10:46.259138 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.259096    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"]
Aug 04 00:10:46.262476 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.262454    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:10:46.271609 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.271581    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"]
Aug 04 00:10:46.312638 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.312608    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7p4g\" (UniqueName: \"kubernetes.io/projected/4f691931-c0a7-489d-926f-deef7759289a-kube-api-access-p7p4g\") pod \"payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8\" (UID: \"4f691931-c0a7-489d-926f-deef7759289a\") " pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:10:46.312815 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.312676    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsjqv\" (UniqueName: \"kubernetes.io/projected/4490ebc2-4f39-4740-9d06-f66577da4d80-kube-api-access-lsjqv\") pod \"payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn\" (UID: \"4490ebc2-4f39-4740-9d06-f66577da4d80\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:10:46.312815 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.312774    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f691931-c0a7-489d-926f-deef7759289a-plugins-config-volume\") pod \"payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8\" (UID: \"4f691931-c0a7-489d-926f-deef7759289a\") " pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:10:46.312815 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.312808    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4490ebc2-4f39-4740-9d06-f66577da4d80-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn\" (UID: \"4490ebc2-4f39-4740-9d06-f66577da4d80\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:10:46.413900 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.413867    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsjqv\" (UniqueName: \"kubernetes.io/projected/4490ebc2-4f39-4740-9d06-f66577da4d80-kube-api-access-lsjqv\") pod \"payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn\" (UID: \"4490ebc2-4f39-4740-9d06-f66577da4d80\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:10:46.414062 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.413933    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f691931-c0a7-489d-926f-deef7759289a-plugins-config-volume\") pod \"payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8\" (UID: \"4f691931-c0a7-489d-926f-deef7759289a\") " pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:10:46.414062 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.413972    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4490ebc2-4f39-4740-9d06-f66577da4d80-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn\" (UID: \"4490ebc2-4f39-4740-9d06-f66577da4d80\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:10:46.414062 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.414027    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7p4g\" (UniqueName: \"kubernetes.io/projected/4f691931-c0a7-489d-926f-deef7759289a-kube-api-access-p7p4g\") pod \"payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8\" (UID: \"4f691931-c0a7-489d-926f-deef7759289a\") " pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:10:46.414698 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.414676    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f691931-c0a7-489d-926f-deef7759289a-plugins-config-volume\") pod \"payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8\" (UID: \"4f691931-c0a7-489d-926f-deef7759289a\") " pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:10:46.414808 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.414730    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4490ebc2-4f39-4740-9d06-f66577da4d80-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn\" (UID: \"4490ebc2-4f39-4740-9d06-f66577da4d80\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:10:46.427546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.427516    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:46.428447 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.428422    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsjqv\" (UniqueName: \"kubernetes.io/projected/4490ebc2-4f39-4740-9d06-f66577da4d80-kube-api-access-lsjqv\") pod \"payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn\" (UID: \"4490ebc2-4f39-4740-9d06-f66577da4d80\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:10:46.428578 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.428489    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7p4g\" (UniqueName: \"kubernetes.io/projected/4f691931-c0a7-489d-926f-deef7759289a-kube-api-access-p7p4g\") pod \"payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8\" (UID: \"4f691931-c0a7-489d-926f-deef7759289a\") " pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:10:46.534077 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.534029    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:10:46.573265 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.573230    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:10:46.581592 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.581566    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"]
Aug 04 00:10:46.584889 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:10:46.584847    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17755e5_5958_446a_8c29_78b736690d0b.slice/crio-f7ff5c3d7461a2e7ab611dd7b3e722569cefd5df17dddc89208c5626d4ed0c80 WatchSource:0}: Error finding container f7ff5c3d7461a2e7ab611dd7b3e722569cefd5df17dddc89208c5626d4ed0c80: Status 404 returned error can't find the container with id f7ff5c3d7461a2e7ab611dd7b3e722569cefd5df17dddc89208c5626d4ed0c80
Aug 04 00:10:46.586494 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.586472    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 00:10:46.689699 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.689663    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"]
Aug 04 00:10:46.692388 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.692354    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx" event={"ID":"c17755e5-5958-446a-8c29-78b736690d0b","Type":"ContainerStarted","Data":"f7ff5c3d7461a2e7ab611dd7b3e722569cefd5df17dddc89208c5626d4ed0c80"}
Aug 04 00:10:46.692509 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:10:46.692456    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f691931_c0a7_489d_926f_deef7759289a.slice/crio-c03186d65c99734c345276742b7c86e959722526472113bc96a71744052b2fea WatchSource:0}: Error finding container c03186d65c99734c345276742b7c86e959722526472113bc96a71744052b2fea: Status 404 returned error can't find the container with id c03186d65c99734c345276742b7c86e959722526472113bc96a71744052b2fea
Aug 04 00:10:46.739378 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:46.739347    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"]
Aug 04 00:10:46.751756 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:10:46.751725    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4490ebc2_4f39_4740_9d06_f66577da4d80.slice/crio-cb141870cb54a363c342c47f2764b63c244e20d03a2de33982b38fce62566d39 WatchSource:0}: Error finding container cb141870cb54a363c342c47f2764b63c244e20d03a2de33982b38fce62566d39: Status 404 returned error can't find the container with id cb141870cb54a363c342c47f2764b63c244e20d03a2de33982b38fce62566d39
Aug 04 00:10:47.699515 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:47.699469    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8" event={"ID":"4f691931-c0a7-489d-926f-deef7759289a","Type":"ContainerStarted","Data":"cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b"}
Aug 04 00:10:47.699996 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:47.699570    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8" event={"ID":"4f691931-c0a7-489d-926f-deef7759289a","Type":"ContainerStarted","Data":"c03186d65c99734c345276742b7c86e959722526472113bc96a71744052b2fea"}
Aug 04 00:10:47.699996 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:47.699689    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:10:47.700991 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:47.700970    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx" event={"ID":"c17755e5-5958-446a-8c29-78b736690d0b","Type":"ContainerStarted","Data":"64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40"}
Aug 04 00:10:47.701057 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:47.701023    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:47.702423 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:47.702398    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn" event={"ID":"4490ebc2-4f39-4740-9d06-f66577da4d80","Type":"ContainerStarted","Data":"a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171"}
Aug 04 00:10:47.702557 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:47.702445    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn" event={"ID":"4490ebc2-4f39-4740-9d06-f66577da4d80","Type":"ContainerStarted","Data":"cb141870cb54a363c342c47f2764b63c244e20d03a2de33982b38fce62566d39"}
Aug 04 00:10:47.702557 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:47.702480    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:10:47.720164 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:47.720123    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8" podStartSLOduration=1.720111062 podStartE2EDuration="1.720111062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:47.718137723 +0000 UTC m=+2455.419844390" watchObservedRunningTime="2026-08-04 00:10:47.720111062 +0000 UTC m=+2455.421817740"
Aug 04 00:10:47.746196 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:47.746146    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn" podStartSLOduration=1.746129191 podStartE2EDuration="1.746129191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:47.745775579 +0000 UTC m=+2455.447482247" watchObservedRunningTime="2026-08-04 00:10:47.746129191 +0000 UTC m=+2455.447835857"
Aug 04 00:10:47.765597 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:47.765548    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx" podStartSLOduration=2.313805397 podStartE2EDuration="2.76553351s" totalImagesPullingTime="451.728113ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:47.763968621 +0000 UTC m=+2455.465675284" watchObservedRunningTime="2026-08-04 00:10:47.76553351 +0000 UTC m=+2455.467240174"
Aug 04 00:10:53.713038 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:53.713011    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:10:58.710097 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:58.710068    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:10:58.710551 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:10:58.710254    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:12:04.481340 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.481309    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"]
Aug 04 00:12:04.481854 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.481522    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx" podUID="c17755e5-5958-446a-8c29-78b736690d0b" containerName="maas-api" containerID="cri-o://64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40" gracePeriod=30
Aug 04 00:12:04.545854 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.545826    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"]
Aug 04 00:12:04.546097 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.546045    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8" podUID="4f691931-c0a7-489d-926f-deef7759289a" containerName="payload-processing" containerID="cri-o://cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b" gracePeriod=30
Aug 04 00:12:04.548416 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.548392    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"]
Aug 04 00:12:04.548641 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.548599    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn" podUID="4490ebc2-4f39-4740-9d06-f66577da4d80" containerName="payload-pre-processing" containerID="cri-o://a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171" gracePeriod=30
Aug 04 00:12:04.807934 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.807904    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:12:04.821890 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.821871    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:12:04.833315 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.833295    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:12:04.927500 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.927470    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c17755e5-5958-446a-8c29-78b736690d0b-openshift-service-ca\") pod \"c17755e5-5958-446a-8c29-78b736690d0b\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") "
Aug 04 00:12:04.927669 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.927525    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4490ebc2-4f39-4740-9d06-f66577da4d80-plugins-config-volume\") pod \"4490ebc2-4f39-4740-9d06-f66577da4d80\" (UID: \"4490ebc2-4f39-4740-9d06-f66577da4d80\") "
Aug 04 00:12:04.927669 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.927552    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsjqv\" (UniqueName: \"kubernetes.io/projected/4490ebc2-4f39-4740-9d06-f66577da4d80-kube-api-access-lsjqv\") pod \"4490ebc2-4f39-4740-9d06-f66577da4d80\" (UID: \"4490ebc2-4f39-4740-9d06-f66577da4d80\") "
Aug 04 00:12:04.927669 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.927579    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfvg7\" (UniqueName: \"kubernetes.io/projected/c17755e5-5958-446a-8c29-78b736690d0b-kube-api-access-cfvg7\") pod \"c17755e5-5958-446a-8c29-78b736690d0b\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") "
Aug 04 00:12:04.927669 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.927609    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c17755e5-5958-446a-8c29-78b736690d0b-maas-api-tls\") pod \"c17755e5-5958-446a-8c29-78b736690d0b\" (UID: \"c17755e5-5958-446a-8c29-78b736690d0b\") "
Aug 04 00:12:04.927932 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.927905    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17755e5-5958-446a-8c29-78b736690d0b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c17755e5-5958-446a-8c29-78b736690d0b" (UID: "c17755e5-5958-446a-8c29-78b736690d0b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:12:04.928014 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.927989    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4490ebc2-4f39-4740-9d06-f66577da4d80-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4490ebc2-4f39-4740-9d06-f66577da4d80" (UID: "4490ebc2-4f39-4740-9d06-f66577da4d80"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:12:04.929924 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.929888    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17755e5-5958-446a-8c29-78b736690d0b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c17755e5-5958-446a-8c29-78b736690d0b" (UID: "c17755e5-5958-446a-8c29-78b736690d0b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:12:04.929924 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.929912    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c17755e5-5958-446a-8c29-78b736690d0b-kube-api-access-cfvg7" (OuterVolumeSpecName: "kube-api-access-cfvg7") pod "c17755e5-5958-446a-8c29-78b736690d0b" (UID: "c17755e5-5958-446a-8c29-78b736690d0b"). InnerVolumeSpecName "kube-api-access-cfvg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:12:04.929924 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:04.929915    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4490ebc2-4f39-4740-9d06-f66577da4d80-kube-api-access-lsjqv" (OuterVolumeSpecName: "kube-api-access-lsjqv") pod "4490ebc2-4f39-4740-9d06-f66577da4d80" (UID: "4490ebc2-4f39-4740-9d06-f66577da4d80"). InnerVolumeSpecName "kube-api-access-lsjqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:12:05.028724 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.028643    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7p4g\" (UniqueName: \"kubernetes.io/projected/4f691931-c0a7-489d-926f-deef7759289a-kube-api-access-p7p4g\") pod \"4f691931-c0a7-489d-926f-deef7759289a\" (UID: \"4f691931-c0a7-489d-926f-deef7759289a\") "
Aug 04 00:12:05.028724 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.028681    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f691931-c0a7-489d-926f-deef7759289a-plugins-config-volume\") pod \"4f691931-c0a7-489d-926f-deef7759289a\" (UID: \"4f691931-c0a7-489d-926f-deef7759289a\") "
Aug 04 00:12:05.029042 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.028993    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c17755e5-5958-446a-8c29-78b736690d0b-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:12:05.029042 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.029025    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4490ebc2-4f39-4740-9d06-f66577da4d80-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:12:05.029042 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.029037    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsjqv\" (UniqueName: \"kubernetes.io/projected/4490ebc2-4f39-4740-9d06-f66577da4d80-kube-api-access-lsjqv\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:12:05.029271 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.029068    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfvg7\" (UniqueName: \"kubernetes.io/projected/c17755e5-5958-446a-8c29-78b736690d0b-kube-api-access-cfvg7\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:12:05.029271 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.029091    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c17755e5-5958-446a-8c29-78b736690d0b-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:12:05.029271 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.029113    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f691931-c0a7-489d-926f-deef7759289a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f691931-c0a7-489d-926f-deef7759289a" (UID: "4f691931-c0a7-489d-926f-deef7759289a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:12:05.030849 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.030822    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f691931-c0a7-489d-926f-deef7759289a-kube-api-access-p7p4g" (OuterVolumeSpecName: "kube-api-access-p7p4g") pod "4f691931-c0a7-489d-926f-deef7759289a" (UID: "4f691931-c0a7-489d-926f-deef7759289a"). InnerVolumeSpecName "kube-api-access-p7p4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:12:05.071340 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.071308    2553 generic.go:356] "Generic (PLEG): container finished" podID="c17755e5-5958-446a-8c29-78b736690d0b" containerID="64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40" exitCode=0
Aug 04 00:12:05.071453 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.071357    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"
Aug 04 00:12:05.071453 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.071389    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx" event={"ID":"c17755e5-5958-446a-8c29-78b736690d0b","Type":"ContainerDied","Data":"64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40"}
Aug 04 00:12:05.071453 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.071434    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx" event={"ID":"c17755e5-5958-446a-8c29-78b736690d0b","Type":"ContainerDied","Data":"f7ff5c3d7461a2e7ab611dd7b3e722569cefd5df17dddc89208c5626d4ed0c80"}
Aug 04 00:12:05.071663 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.071457    2553 scope.go:117] "RemoveContainer" containerID="64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40"
Aug 04 00:12:05.081209 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.081133    2553 generic.go:356] "Generic (PLEG): container finished" podID="4490ebc2-4f39-4740-9d06-f66577da4d80" containerID="a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171" exitCode=0
Aug 04 00:12:05.081486 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.081462    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"
Aug 04 00:12:05.081649 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.081216    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn" event={"ID":"4490ebc2-4f39-4740-9d06-f66577da4d80","Type":"ContainerDied","Data":"a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171"}
Aug 04 00:12:05.081867 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.081653    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn" event={"ID":"4490ebc2-4f39-4740-9d06-f66577da4d80","Type":"ContainerDied","Data":"cb141870cb54a363c342c47f2764b63c244e20d03a2de33982b38fce62566d39"}
Aug 04 00:12:05.083775 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.083750    2553 generic.go:356] "Generic (PLEG): container finished" podID="4f691931-c0a7-489d-926f-deef7759289a" containerID="cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b" exitCode=0
Aug 04 00:12:05.083868 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.083787    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8" event={"ID":"4f691931-c0a7-489d-926f-deef7759289a","Type":"ContainerDied","Data":"cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b"}
Aug 04 00:12:05.083868 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.083816    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"
Aug 04 00:12:05.083868 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.083837    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8" event={"ID":"4f691931-c0a7-489d-926f-deef7759289a","Type":"ContainerDied","Data":"c03186d65c99734c345276742b7c86e959722526472113bc96a71744052b2fea"}
Aug 04 00:12:05.090349 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.090331    2553 scope.go:117] "RemoveContainer" containerID="64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40"
Aug 04 00:12:05.090685 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:12:05.090662    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40\": container with ID starting with 64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40 not found: ID does not exist" containerID="64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40"
Aug 04 00:12:05.090774 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.090697    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40"} err="failed to get container status \"64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40\": rpc error: code = NotFound desc = could not find container \"64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40\": container with ID starting with 64a67512a58f8c27d627ea441d277b1ba166aeef973350ad93b724848b551e40 not found: ID does not exist"
Aug 04 00:12:05.090774 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.090750    2553 scope.go:117] "RemoveContainer" containerID="a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171"
Aug 04 00:12:05.099106 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.099091    2553 scope.go:117] "RemoveContainer" containerID="a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171"
Aug 04 00:12:05.099334 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:12:05.099315    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171\": container with ID starting with a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171 not found: ID does not exist" containerID="a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171"
Aug 04 00:12:05.099412 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.099340    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171"} err="failed to get container status \"a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171\": rpc error: code = NotFound desc = could not find container \"a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171\": container with ID starting with a7dca695f66907495f6aead1a032b72bb9ad82f4f091ea05a695ed9f021be171 not found: ID does not exist"
Aug 04 00:12:05.099412 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.099356    2553 scope.go:117] "RemoveContainer" containerID="cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b"
Aug 04 00:12:05.107025 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.107005    2553 scope.go:117] "RemoveContainer" containerID="cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b"
Aug 04 00:12:05.107434 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:12:05.107415    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b\": container with ID starting with cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b not found: ID does not exist" containerID="cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b"
Aug 04 00:12:05.107502 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.107438    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b"} err="failed to get container status \"cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b\": rpc error: code = NotFound desc = could not find container \"cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b\": container with ID starting with cd8de24eff1d4baf577711367607509eabf74044c2807cd11164e940ff34732b not found: ID does not exist"
Aug 04 00:12:05.109213 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.109193    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"]
Aug 04 00:12:05.114820 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.114802    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-335654eb-6f88766bdf-w6rfx"]
Aug 04 00:12:05.130158 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.130123    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7p4g\" (UniqueName: \"kubernetes.io/projected/4f691931-c0a7-489d-926f-deef7759289a-kube-api-access-p7p4g\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:12:05.130235 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.130160    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f691931-c0a7-489d-926f-deef7759289a-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:12:05.132583 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.132562    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"]
Aug 04 00:12:05.135757 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.135738    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-335654eb-5cd5c5bfd-7q8m8"]
Aug 04 00:12:05.148466 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.148443    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"]
Aug 04 00:12:05.153593 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:05.153572    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-335654eb-7b5c4b6f6-jccqn"]
Aug 04 00:12:06.765813 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:06.765773    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4490ebc2-4f39-4740-9d06-f66577da4d80" path="/var/lib/kubelet/pods/4490ebc2-4f39-4740-9d06-f66577da4d80/volumes"
Aug 04 00:12:06.766321 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:06.766302    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f691931-c0a7-489d-926f-deef7759289a" path="/var/lib/kubelet/pods/4f691931-c0a7-489d-926f-deef7759289a/volumes"
Aug 04 00:12:06.766883 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:12:06.766865    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c17755e5-5958-446a-8c29-78b736690d0b" path="/var/lib/kubelet/pods/c17755e5-5958-446a-8c29-78b736690d0b/volumes"
Aug 04 00:14:07.472361 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.472324    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"]
Aug 04 00:14:07.473071 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.473047    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f691931-c0a7-489d-926f-deef7759289a" containerName="payload-processing"
Aug 04 00:14:07.473149 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.473074    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f691931-c0a7-489d-926f-deef7759289a" containerName="payload-processing"
Aug 04 00:14:07.473149 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.473094    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17755e5-5958-446a-8c29-78b736690d0b" containerName="maas-api"
Aug 04 00:14:07.473149 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.473103    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17755e5-5958-446a-8c29-78b736690d0b" containerName="maas-api"
Aug 04 00:14:07.473149 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.473142    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4490ebc2-4f39-4740-9d06-f66577da4d80" containerName="payload-pre-processing"
Aug 04 00:14:07.473367 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.473151    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4490ebc2-4f39-4740-9d06-f66577da4d80" containerName="payload-pre-processing"
Aug 04 00:14:07.473367 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.473237    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="4490ebc2-4f39-4740-9d06-f66577da4d80" containerName="payload-pre-processing"
Aug 04 00:14:07.473367 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.473249    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17755e5-5958-446a-8c29-78b736690d0b" containerName="maas-api"
Aug 04 00:14:07.473367 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.473260    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f691931-c0a7-489d-926f-deef7759289a" containerName="payload-processing"
Aug 04 00:14:07.476760 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.476736    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:07.479219 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.479197    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e43d99-serving-cert\""
Aug 04 00:14:07.486207 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.486149    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"]
Aug 04 00:14:07.558548 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.558514    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e337fae8-20d1-41c8-bbf4-b027f03b289a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:07.558764 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.558559    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svs6h\" (UniqueName: \"kubernetes.io/projected/e337fae8-20d1-41c8-bbf4-b027f03b289a-kube-api-access-svs6h\") pod \"maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:07.558764 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.558631    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e337fae8-20d1-41c8-bbf4-b027f03b289a-maas-api-tls\") pod \"maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:07.659606 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.659571    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e337fae8-20d1-41c8-bbf4-b027f03b289a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:07.659801 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.659622    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svs6h\" (UniqueName: \"kubernetes.io/projected/e337fae8-20d1-41c8-bbf4-b027f03b289a-kube-api-access-svs6h\") pod \"maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:07.659801 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.659668    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e337fae8-20d1-41c8-bbf4-b027f03b289a-maas-api-tls\") pod \"maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:07.660228 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.660207    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e337fae8-20d1-41c8-bbf4-b027f03b289a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:07.662174 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.662152    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e337fae8-20d1-41c8-bbf4-b027f03b289a-maas-api-tls\") pod \"maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:07.671687 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.671669    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svs6h\" (UniqueName: \"kubernetes.io/projected/e337fae8-20d1-41c8-bbf4-b027f03b289a-kube-api-access-svs6h\") pod \"maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:07.790173 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.790108    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:07.924362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:07.924337    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"]
Aug 04 00:14:07.927613 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:14:07.927583    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode337fae8_20d1_41c8_bbf4_b027f03b289a.slice/crio-126c51cbc60f853c6ec7b1b7f2de4d62e9f0ec31a564ef3b44c548a3ab0f95bf WatchSource:0}: Error finding container 126c51cbc60f853c6ec7b1b7f2de4d62e9f0ec31a564ef3b44c548a3ab0f95bf: Status 404 returned error can't find the container with id 126c51cbc60f853c6ec7b1b7f2de4d62e9f0ec31a564ef3b44c548a3ab0f95bf
Aug 04 00:14:08.183378 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.183347    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"]
Aug 04 00:14:08.188032 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.188007    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:14:08.190524 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.190502    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e43d99-dockercfg-xjnjw\""
Aug 04 00:14:08.190667 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.190610    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e43d99\""
Aug 04 00:14:08.195774 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.195752    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"]
Aug 04 00:14:08.265411 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.265385    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2rwb\" (UniqueName: \"kubernetes.io/projected/4c859e1c-ee84-4fed-8784-0e99b944b8f5-kube-api-access-k2rwb\") pod \"payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6\" (UID: \"4c859e1c-ee84-4fed-8784-0e99b944b8f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:14:08.265535 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.265421    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c859e1c-ee84-4fed-8784-0e99b944b8f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6\" (UID: \"4c859e1c-ee84-4fed-8784-0e99b944b8f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:14:08.315537 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.315513    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"]
Aug 04 00:14:08.319131 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.319115    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:14:08.328273 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.328199    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"]
Aug 04 00:14:08.366640 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.366609    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7301fa45-6478-4861-b120-3b37dd3c9845-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j\" (UID: \"7301fa45-6478-4861-b120-3b37dd3c9845\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:14:08.366786 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.366657    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b29jx\" (UniqueName: \"kubernetes.io/projected/7301fa45-6478-4861-b120-3b37dd3c9845-kube-api-access-b29jx\") pod \"payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j\" (UID: \"7301fa45-6478-4861-b120-3b37dd3c9845\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:14:08.366786 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.366750    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2rwb\" (UniqueName: \"kubernetes.io/projected/4c859e1c-ee84-4fed-8784-0e99b944b8f5-kube-api-access-k2rwb\") pod \"payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6\" (UID: \"4c859e1c-ee84-4fed-8784-0e99b944b8f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:14:08.366882 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.366823    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c859e1c-ee84-4fed-8784-0e99b944b8f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6\" (UID: \"4c859e1c-ee84-4fed-8784-0e99b944b8f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:14:08.367428 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.367412    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c859e1c-ee84-4fed-8784-0e99b944b8f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6\" (UID: \"4c859e1c-ee84-4fed-8784-0e99b944b8f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:14:08.374533 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.374506    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2rwb\" (UniqueName: \"kubernetes.io/projected/4c859e1c-ee84-4fed-8784-0e99b944b8f5-kube-api-access-k2rwb\") pod \"payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6\" (UID: \"4c859e1c-ee84-4fed-8784-0e99b944b8f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:14:08.468359 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.468267    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b29jx\" (UniqueName: \"kubernetes.io/projected/7301fa45-6478-4861-b120-3b37dd3c9845-kube-api-access-b29jx\") pod \"payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j\" (UID: \"7301fa45-6478-4861-b120-3b37dd3c9845\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:14:08.468441 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.468391    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7301fa45-6478-4861-b120-3b37dd3c9845-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j\" (UID: \"7301fa45-6478-4861-b120-3b37dd3c9845\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:14:08.469226 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.469127    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7301fa45-6478-4861-b120-3b37dd3c9845-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j\" (UID: \"7301fa45-6478-4861-b120-3b37dd3c9845\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:14:08.478488 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.478468    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b29jx\" (UniqueName: \"kubernetes.io/projected/7301fa45-6478-4861-b120-3b37dd3c9845-kube-api-access-b29jx\") pod \"payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j\" (UID: \"7301fa45-6478-4861-b120-3b37dd3c9845\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:14:08.501435 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.501403    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:14:08.629420 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.629387    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:14:08.675896 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.675854    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5" event={"ID":"e337fae8-20d1-41c8-bbf4-b027f03b289a","Type":"ContainerStarted","Data":"8b5c4ef7895ff4712717e1d9356fdda8f7fbb52e7d4614d04c647328683294b0"}
Aug 04 00:14:08.676058 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.675916    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5" event={"ID":"e337fae8-20d1-41c8-bbf4-b027f03b289a","Type":"ContainerStarted","Data":"126c51cbc60f853c6ec7b1b7f2de4d62e9f0ec31a564ef3b44c548a3ab0f95bf"}
Aug 04 00:14:08.676355 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.676317    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:08.695203 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.695156    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5" podStartSLOduration=1.156025975 podStartE2EDuration="1.695140371s" totalImagesPullingTime="539.114396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:14:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:14:08.691880274 +0000 UTC m=+2656.393586940" watchObservedRunningTime="2026-08-04 00:14:08.695140371 +0000 UTC m=+2656.396847036"
Aug 04 00:14:08.769693 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.769664    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"]
Aug 04 00:14:08.771594 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:14:08.771291    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7301fa45_6478_4861_b120_3b37dd3c9845.slice/crio-a88f14de753e82f497970f54aa4c6bfa3ed8c93e65d73ef4f794f58bf47914f0 WatchSource:0}: Error finding container a88f14de753e82f497970f54aa4c6bfa3ed8c93e65d73ef4f794f58bf47914f0: Status 404 returned error can't find the container with id a88f14de753e82f497970f54aa4c6bfa3ed8c93e65d73ef4f794f58bf47914f0
Aug 04 00:14:08.845725 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:08.845683    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"]
Aug 04 00:14:08.847454 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:14:08.847431    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c859e1c_ee84_4fed_8784_0e99b944b8f5.slice/crio-ffbfa64d36470ede9418c374efed581cf242a123d07526f5caaa58225cee3e7d WatchSource:0}: Error finding container ffbfa64d36470ede9418c374efed581cf242a123d07526f5caaa58225cee3e7d: Status 404 returned error can't find the container with id ffbfa64d36470ede9418c374efed581cf242a123d07526f5caaa58225cee3e7d
Aug 04 00:14:09.681916 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:09.681878    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j" event={"ID":"7301fa45-6478-4861-b120-3b37dd3c9845","Type":"ContainerStarted","Data":"abfa83f7502f92e5d8b1dd5b03d0eeae5ca9297505d1b0bad1b35b8660354ed5"}
Aug 04 00:14:09.682352 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:09.681941    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j" event={"ID":"7301fa45-6478-4861-b120-3b37dd3c9845","Type":"ContainerStarted","Data":"a88f14de753e82f497970f54aa4c6bfa3ed8c93e65d73ef4f794f58bf47914f0"}
Aug 04 00:14:09.682352 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:09.682044    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:14:09.683576 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:09.683556    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6" event={"ID":"4c859e1c-ee84-4fed-8784-0e99b944b8f5","Type":"ContainerStarted","Data":"2335847873a6b1b584e5d99289bedf9db93fe519c708b7e3ff2678205887a0d5"}
Aug 04 00:14:09.683938 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:09.683837    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6" event={"ID":"4c859e1c-ee84-4fed-8784-0e99b944b8f5","Type":"ContainerStarted","Data":"ffbfa64d36470ede9418c374efed581cf242a123d07526f5caaa58225cee3e7d"}
Aug 04 00:14:09.684682 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:09.684647    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:14:09.698315 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:09.698271    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j" podStartSLOduration=1.6982579599999998 podStartE2EDuration="1.69825796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:14:09.69602201 +0000 UTC m=+2657.397728670" watchObservedRunningTime="2026-08-04 00:14:09.69825796 +0000 UTC m=+2657.399964624"
Aug 04 00:14:09.712808 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:09.712767    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6" podStartSLOduration=1.712754624 podStartE2EDuration="1.712754624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:14:09.710730629 +0000 UTC m=+2657.412437293" watchObservedRunningTime="2026-08-04 00:14:09.712754624 +0000 UTC m=+2657.414461287"
Aug 04 00:14:14.690410 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:14.690376    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:14:20.690499 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:20.690472    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:14:21.694799 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:21.694767    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:14:35.785358 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:35.785318    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"]
Aug 04 00:14:35.789307 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:35.789280    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:35.791743 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:35.791720    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b0f5b6-serving-cert\""
Aug 04 00:14:35.797752 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:35.797696    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"]
Aug 04 00:14:35.915586 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:35.915552    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7055a49-ebe2-4868-9d17-f949a51ae86f-maas-api-tls\") pod \"maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:35.915769 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:35.915595    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gbpf\" (UniqueName: \"kubernetes.io/projected/c7055a49-ebe2-4868-9d17-f949a51ae86f-kube-api-access-2gbpf\") pod \"maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:35.915769 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:35.915632    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7055a49-ebe2-4868-9d17-f949a51ae86f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:36.016404 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.016379    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7055a49-ebe2-4868-9d17-f949a51ae86f-maas-api-tls\") pod \"maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:36.016558 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.016414    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gbpf\" (UniqueName: \"kubernetes.io/projected/c7055a49-ebe2-4868-9d17-f949a51ae86f-kube-api-access-2gbpf\") pod \"maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:36.016558 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.016440    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7055a49-ebe2-4868-9d17-f949a51ae86f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:36.017079 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.017055    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7055a49-ebe2-4868-9d17-f949a51ae86f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:36.018939 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.018918    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7055a49-ebe2-4868-9d17-f949a51ae86f-maas-api-tls\") pod \"maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:36.026407 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.026387    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gbpf\" (UniqueName: \"kubernetes.io/projected/c7055a49-ebe2-4868-9d17-f949a51ae86f-kube-api-access-2gbpf\") pod \"maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:36.104037 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.103959    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:36.238131 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.238106    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"]
Aug 04 00:14:36.240062 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:14:36.240037    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7055a49_ebe2_4868_9d17_f949a51ae86f.slice/crio-57312a3f325e7862bb44aa93d19dd539387b2ccafbe91e876bb859ba10ac69a8 WatchSource:0}: Error finding container 57312a3f325e7862bb44aa93d19dd539387b2ccafbe91e876bb859ba10ac69a8: Status 404 returned error can't find the container with id 57312a3f325e7862bb44aa93d19dd539387b2ccafbe91e876bb859ba10ac69a8
Aug 04 00:14:36.489974 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.489941    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"]
Aug 04 00:14:36.494528 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.494508    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:14:36.497135 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.497116    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-b0f5b6-dockercfg-whgzm\""
Aug 04 00:14:36.497230 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.497174    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-b0f5b6\""
Aug 04 00:14:36.504940 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.504921    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"]
Aug 04 00:14:36.531061 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.531034    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"]
Aug 04 00:14:36.535070 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.535053    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:14:36.542853 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.542834    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"]
Aug 04 00:14:36.623511 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.623483    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f5f4966-0712-4f40-b5a1-365e889e0dd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92\" (UID: \"5f5f4966-0712-4f40-b5a1-365e889e0dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:14:36.623673 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.623530    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d3ce8b9-850f-48d5-ad02-87758cf50b24-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p\" (UID: \"9d3ce8b9-850f-48d5-ad02-87758cf50b24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:14:36.623673 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.623559    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtzs5\" (UniqueName: \"kubernetes.io/projected/5f5f4966-0712-4f40-b5a1-365e889e0dd6-kube-api-access-wtzs5\") pod \"payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92\" (UID: \"5f5f4966-0712-4f40-b5a1-365e889e0dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:14:36.623673 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.623622    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txgwz\" (UniqueName: \"kubernetes.io/projected/9d3ce8b9-850f-48d5-ad02-87758cf50b24-kube-api-access-txgwz\") pod \"payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p\" (UID: \"9d3ce8b9-850f-48d5-ad02-87758cf50b24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:14:36.725040 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.724995    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f5f4966-0712-4f40-b5a1-365e889e0dd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92\" (UID: \"5f5f4966-0712-4f40-b5a1-365e889e0dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:14:36.725145 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.725062    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d3ce8b9-850f-48d5-ad02-87758cf50b24-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p\" (UID: \"9d3ce8b9-850f-48d5-ad02-87758cf50b24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:14:36.725145 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.725095    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtzs5\" (UniqueName: \"kubernetes.io/projected/5f5f4966-0712-4f40-b5a1-365e889e0dd6-kube-api-access-wtzs5\") pod \"payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92\" (UID: \"5f5f4966-0712-4f40-b5a1-365e889e0dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:14:36.725145 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.725121    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txgwz\" (UniqueName: \"kubernetes.io/projected/9d3ce8b9-850f-48d5-ad02-87758cf50b24-kube-api-access-txgwz\") pod \"payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p\" (UID: \"9d3ce8b9-850f-48d5-ad02-87758cf50b24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:14:36.725652 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.725634    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f5f4966-0712-4f40-b5a1-365e889e0dd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92\" (UID: \"5f5f4966-0712-4f40-b5a1-365e889e0dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:14:36.725805 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.725790    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d3ce8b9-850f-48d5-ad02-87758cf50b24-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p\" (UID: \"9d3ce8b9-850f-48d5-ad02-87758cf50b24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:14:36.733362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.733339    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtzs5\" (UniqueName: \"kubernetes.io/projected/5f5f4966-0712-4f40-b5a1-365e889e0dd6-kube-api-access-wtzs5\") pod \"payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92\" (UID: \"5f5f4966-0712-4f40-b5a1-365e889e0dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:14:36.733543 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.733524    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txgwz\" (UniqueName: \"kubernetes.io/projected/9d3ce8b9-850f-48d5-ad02-87758cf50b24-kube-api-access-txgwz\") pod \"payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p\" (UID: \"9d3ce8b9-850f-48d5-ad02-87758cf50b24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:14:36.806109 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.806040    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:14:36.816270 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.816240    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64" event={"ID":"c7055a49-ebe2-4868-9d17-f949a51ae86f","Type":"ContainerStarted","Data":"00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2"}
Aug 04 00:14:36.816360 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.816317    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:36.816360 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.816346    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64" event={"ID":"c7055a49-ebe2-4868-9d17-f949a51ae86f","Type":"ContainerStarted","Data":"57312a3f325e7862bb44aa93d19dd539387b2ccafbe91e876bb859ba10ac69a8"}
Aug 04 00:14:36.834615 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.834565    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64" podStartSLOduration=1.440508717 podStartE2EDuration="1.834549791s" totalImagesPullingTime="394.041074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:14:36.832258888 +0000 UTC m=+2684.533965553" watchObservedRunningTime="2026-08-04 00:14:36.834549791 +0000 UTC m=+2684.536256454"
Aug 04 00:14:36.846006 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.845983    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:14:36.951745 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.951695    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"]
Aug 04 00:14:36.954064 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:14:36.954035    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5f4966_0712_4f40_b5a1_365e889e0dd6.slice/crio-907d7aaa23465ec328dfe611645b3b624a66bc35f1aa5907196a09b43de81a30 WatchSource:0}: Error finding container 907d7aaa23465ec328dfe611645b3b624a66bc35f1aa5907196a09b43de81a30: Status 404 returned error can't find the container with id 907d7aaa23465ec328dfe611645b3b624a66bc35f1aa5907196a09b43de81a30
Aug 04 00:14:36.999022 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:36.998999    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"]
Aug 04 00:14:37.003856 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:14:37.003827    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d3ce8b9_850f_48d5_ad02_87758cf50b24.slice/crio-45b0131ee360be012f6d00a78eaef4c67c8205f63ec44e77481e572486faad07 WatchSource:0}: Error finding container 45b0131ee360be012f6d00a78eaef4c67c8205f63ec44e77481e572486faad07: Status 404 returned error can't find the container with id 45b0131ee360be012f6d00a78eaef4c67c8205f63ec44e77481e572486faad07
Aug 04 00:14:37.823501 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:37.823453    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p" event={"ID":"9d3ce8b9-850f-48d5-ad02-87758cf50b24","Type":"ContainerStarted","Data":"97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b"}
Aug 04 00:14:37.823978 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:37.823552    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:14:37.823978 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:37.823577    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p" event={"ID":"9d3ce8b9-850f-48d5-ad02-87758cf50b24","Type":"ContainerStarted","Data":"45b0131ee360be012f6d00a78eaef4c67c8205f63ec44e77481e572486faad07"}
Aug 04 00:14:37.825097 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:37.825067    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92" event={"ID":"5f5f4966-0712-4f40-b5a1-365e889e0dd6","Type":"ContainerStarted","Data":"4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622"}
Aug 04 00:14:37.825226 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:37.825116    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92" event={"ID":"5f5f4966-0712-4f40-b5a1-365e889e0dd6","Type":"ContainerStarted","Data":"907d7aaa23465ec328dfe611645b3b624a66bc35f1aa5907196a09b43de81a30"}
Aug 04 00:14:37.825226 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:37.825145    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:14:37.840915 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:37.840853    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p" podStartSLOduration=1.8408348810000001 podStartE2EDuration="1.840834881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:14:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:14:37.837547777 +0000 UTC m=+2685.539254443" watchObservedRunningTime="2026-08-04 00:14:37.840834881 +0000 UTC m=+2685.542541549"
Aug 04 00:14:37.852630 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:37.852574    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92" podStartSLOduration=1.8525606780000001 podStartE2EDuration="1.852560678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:14:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:14:37.851559137 +0000 UTC m=+2685.553265800" watchObservedRunningTime="2026-08-04 00:14:37.852560678 +0000 UTC m=+2685.554267340"
Aug 04 00:14:42.830284 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:42.830258    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:14:48.833150 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:48.833121    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:14:48.833610 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:14:48.833277    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:16:20.014923 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.014832    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"]
Aug 04 00:16:20.015509 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.015082    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5" podUID="e337fae8-20d1-41c8-bbf4-b027f03b289a" containerName="maas-api" containerID="cri-o://8b5c4ef7895ff4712717e1d9356fdda8f7fbb52e7d4614d04c647328683294b0" gracePeriod=30
Aug 04 00:16:20.067762 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.067730    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"]
Aug 04 00:16:20.068025 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.067984    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j" podUID="7301fa45-6478-4861-b120-3b37dd3c9845" containerName="payload-processing" containerID="cri-o://abfa83f7502f92e5d8b1dd5b03d0eeae5ca9297505d1b0bad1b35b8660354ed5" gracePeriod=30
Aug 04 00:16:20.073542 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.073512    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"]
Aug 04 00:16:20.073871 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.073775    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6" podUID="4c859e1c-ee84-4fed-8784-0e99b944b8f5" containerName="payload-pre-processing" containerID="cri-o://2335847873a6b1b584e5d99289bedf9db93fe519c708b7e3ff2678205887a0d5" gracePeriod=30
Aug 04 00:16:20.349784 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.349753    2553 generic.go:356] "Generic (PLEG): container finished" podID="e337fae8-20d1-41c8-bbf4-b027f03b289a" containerID="8b5c4ef7895ff4712717e1d9356fdda8f7fbb52e7d4614d04c647328683294b0" exitCode=0
Aug 04 00:16:20.349957 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.349825    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5" event={"ID":"e337fae8-20d1-41c8-bbf4-b027f03b289a","Type":"ContainerDied","Data":"8b5c4ef7895ff4712717e1d9356fdda8f7fbb52e7d4614d04c647328683294b0"}
Aug 04 00:16:20.351246 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.351222    2553 generic.go:356] "Generic (PLEG): container finished" podID="7301fa45-6478-4861-b120-3b37dd3c9845" containerID="abfa83f7502f92e5d8b1dd5b03d0eeae5ca9297505d1b0bad1b35b8660354ed5" exitCode=0
Aug 04 00:16:20.351463 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.351312    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j" event={"ID":"7301fa45-6478-4861-b120-3b37dd3c9845","Type":"ContainerDied","Data":"abfa83f7502f92e5d8b1dd5b03d0eeae5ca9297505d1b0bad1b35b8660354ed5"}
Aug 04 00:16:20.351463 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.351366    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j" event={"ID":"7301fa45-6478-4861-b120-3b37dd3c9845","Type":"ContainerDied","Data":"a88f14de753e82f497970f54aa4c6bfa3ed8c93e65d73ef4f794f58bf47914f0"}
Aug 04 00:16:20.351463 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.351391    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a88f14de753e82f497970f54aa4c6bfa3ed8c93e65d73ef4f794f58bf47914f0"
Aug 04 00:16:20.352839 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.352799    2553 generic.go:356] "Generic (PLEG): container finished" podID="4c859e1c-ee84-4fed-8784-0e99b944b8f5" containerID="2335847873a6b1b584e5d99289bedf9db93fe519c708b7e3ff2678205887a0d5" exitCode=0
Aug 04 00:16:20.353021 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.352849    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6" event={"ID":"4c859e1c-ee84-4fed-8784-0e99b944b8f5","Type":"ContainerDied","Data":"2335847873a6b1b584e5d99289bedf9db93fe519c708b7e3ff2678205887a0d5"}
Aug 04 00:16:20.353021 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.352894    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6" event={"ID":"4c859e1c-ee84-4fed-8784-0e99b944b8f5","Type":"ContainerDied","Data":"ffbfa64d36470ede9418c374efed581cf242a123d07526f5caaa58225cee3e7d"}
Aug 04 00:16:20.353021 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.352922    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ffbfa64d36470ede9418c374efed581cf242a123d07526f5caaa58225cee3e7d"
Aug 04 00:16:20.365381 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.365355    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:16:20.368895 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.368836    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:16:20.502337 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.502311    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7301fa45-6478-4861-b120-3b37dd3c9845-plugins-config-volume\") pod \"7301fa45-6478-4861-b120-3b37dd3c9845\" (UID: \"7301fa45-6478-4861-b120-3b37dd3c9845\") "
Aug 04 00:16:20.502491 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.502343    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c859e1c-ee84-4fed-8784-0e99b944b8f5-plugins-config-volume\") pod \"4c859e1c-ee84-4fed-8784-0e99b944b8f5\" (UID: \"4c859e1c-ee84-4fed-8784-0e99b944b8f5\") "
Aug 04 00:16:20.502491 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.502393    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2rwb\" (UniqueName: \"kubernetes.io/projected/4c859e1c-ee84-4fed-8784-0e99b944b8f5-kube-api-access-k2rwb\") pod \"4c859e1c-ee84-4fed-8784-0e99b944b8f5\" (UID: \"4c859e1c-ee84-4fed-8784-0e99b944b8f5\") "
Aug 04 00:16:20.502491 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.502414    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b29jx\" (UniqueName: \"kubernetes.io/projected/7301fa45-6478-4861-b120-3b37dd3c9845-kube-api-access-b29jx\") pod \"7301fa45-6478-4861-b120-3b37dd3c9845\" (UID: \"7301fa45-6478-4861-b120-3b37dd3c9845\") "
Aug 04 00:16:20.502658 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.502637    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7301fa45-6478-4861-b120-3b37dd3c9845-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7301fa45-6478-4861-b120-3b37dd3c9845" (UID: "7301fa45-6478-4861-b120-3b37dd3c9845"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:16:20.502753 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.502730    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c859e1c-ee84-4fed-8784-0e99b944b8f5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4c859e1c-ee84-4fed-8784-0e99b944b8f5" (UID: "4c859e1c-ee84-4fed-8784-0e99b944b8f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:16:20.503348 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.503329    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:16:20.504817 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.504795    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7301fa45-6478-4861-b120-3b37dd3c9845-kube-api-access-b29jx" (OuterVolumeSpecName: "kube-api-access-b29jx") pod "7301fa45-6478-4861-b120-3b37dd3c9845" (UID: "7301fa45-6478-4861-b120-3b37dd3c9845"). InnerVolumeSpecName "kube-api-access-b29jx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:16:20.504896 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.504827    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c859e1c-ee84-4fed-8784-0e99b944b8f5-kube-api-access-k2rwb" (OuterVolumeSpecName: "kube-api-access-k2rwb") pod "4c859e1c-ee84-4fed-8784-0e99b944b8f5" (UID: "4c859e1c-ee84-4fed-8784-0e99b944b8f5"). InnerVolumeSpecName "kube-api-access-k2rwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:16:20.604018 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.603919    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svs6h\" (UniqueName: \"kubernetes.io/projected/e337fae8-20d1-41c8-bbf4-b027f03b289a-kube-api-access-svs6h\") pod \"e337fae8-20d1-41c8-bbf4-b027f03b289a\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") "
Aug 04 00:16:20.604018 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.604017    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e337fae8-20d1-41c8-bbf4-b027f03b289a-openshift-service-ca\") pod \"e337fae8-20d1-41c8-bbf4-b027f03b289a\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") "
Aug 04 00:16:20.604211 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.604065    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e337fae8-20d1-41c8-bbf4-b027f03b289a-maas-api-tls\") pod \"e337fae8-20d1-41c8-bbf4-b027f03b289a\" (UID: \"e337fae8-20d1-41c8-bbf4-b027f03b289a\") "
Aug 04 00:16:20.604395 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.604367    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7301fa45-6478-4861-b120-3b37dd3c9845-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:20.604395 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.604382    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e337fae8-20d1-41c8-bbf4-b027f03b289a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e337fae8-20d1-41c8-bbf4-b027f03b289a" (UID: "e337fae8-20d1-41c8-bbf4-b027f03b289a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:16:20.604560 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.604398    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c859e1c-ee84-4fed-8784-0e99b944b8f5-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:20.604560 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.604437    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2rwb\" (UniqueName: \"kubernetes.io/projected/4c859e1c-ee84-4fed-8784-0e99b944b8f5-kube-api-access-k2rwb\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:20.604560 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.604454    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b29jx\" (UniqueName: \"kubernetes.io/projected/7301fa45-6478-4861-b120-3b37dd3c9845-kube-api-access-b29jx\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:20.606246 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.606223    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e337fae8-20d1-41c8-bbf4-b027f03b289a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e337fae8-20d1-41c8-bbf4-b027f03b289a" (UID: "e337fae8-20d1-41c8-bbf4-b027f03b289a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:16:20.606246 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.606229    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e337fae8-20d1-41c8-bbf4-b027f03b289a-kube-api-access-svs6h" (OuterVolumeSpecName: "kube-api-access-svs6h") pod "e337fae8-20d1-41c8-bbf4-b027f03b289a" (UID: "e337fae8-20d1-41c8-bbf4-b027f03b289a"). InnerVolumeSpecName "kube-api-access-svs6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:16:20.705944 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.705908    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e337fae8-20d1-41c8-bbf4-b027f03b289a-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:20.705944 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.705941    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e337fae8-20d1-41c8-bbf4-b027f03b289a-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:20.705944 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:20.705952    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svs6h\" (UniqueName: \"kubernetes.io/projected/e337fae8-20d1-41c8-bbf4-b027f03b289a-kube-api-access-svs6h\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:21.359914 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:21.359822    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"
Aug 04 00:16:21.359914 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:21.359872    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5" event={"ID":"e337fae8-20d1-41c8-bbf4-b027f03b289a","Type":"ContainerDied","Data":"126c51cbc60f853c6ec7b1b7f2de4d62e9f0ec31a564ef3b44c548a3ab0f95bf"}
Aug 04 00:16:21.360416 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:21.359942    2553 scope.go:117] "RemoveContainer" containerID="8b5c4ef7895ff4712717e1d9356fdda8f7fbb52e7d4614d04c647328683294b0"
Aug 04 00:16:21.360416 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:21.359967    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"
Aug 04 00:16:21.360416 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:21.360164    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"
Aug 04 00:16:21.382720 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:21.382679    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"]
Aug 04 00:16:21.386470 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:21.386444    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e43d99-d64ddf5d-8k8k5"]
Aug 04 00:16:21.396255 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:21.396228    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"]
Aug 04 00:16:21.399559 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:21.399531    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e43d99-8697695488-frxq6"]
Aug 04 00:16:21.411495 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:21.411461    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"]
Aug 04 00:16:21.418641 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:21.418610    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e43d99-55b8498858-gqn5j"]
Aug 04 00:16:22.765147 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:22.765116    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c859e1c-ee84-4fed-8784-0e99b944b8f5" path="/var/lib/kubelet/pods/4c859e1c-ee84-4fed-8784-0e99b944b8f5/volumes"
Aug 04 00:16:22.765518 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:22.765464    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7301fa45-6478-4861-b120-3b37dd3c9845" path="/var/lib/kubelet/pods/7301fa45-6478-4861-b120-3b37dd3c9845/volumes"
Aug 04 00:16:22.765812 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:22.765799    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e337fae8-20d1-41c8-bbf4-b027f03b289a" path="/var/lib/kubelet/pods/e337fae8-20d1-41c8-bbf4-b027f03b289a/volumes"
Aug 04 00:16:37.898368 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:37.898331    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"]
Aug 04 00:16:37.898795 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:37.898601    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64" podUID="c7055a49-ebe2-4868-9d17-f949a51ae86f" containerName="maas-api" containerID="cri-o://00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2" gracePeriod=30
Aug 04 00:16:37.950767 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:37.950732    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"]
Aug 04 00:16:37.950986 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:37.950959    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p" podUID="9d3ce8b9-850f-48d5-ad02-87758cf50b24" containerName="payload-processing" containerID="cri-o://97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b" gracePeriod=30
Aug 04 00:16:37.958564 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:37.958535    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"]
Aug 04 00:16:37.958821 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:37.958772    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92" podUID="5f5f4966-0712-4f40-b5a1-365e889e0dd6" containerName="payload-pre-processing" containerID="cri-o://4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622" gracePeriod=30
Aug 04 00:16:38.230252 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.230225    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:16:38.232265 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.232246    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:16:38.256585 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.256562    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:16:38.376368 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.376332    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txgwz\" (UniqueName: \"kubernetes.io/projected/9d3ce8b9-850f-48d5-ad02-87758cf50b24-kube-api-access-txgwz\") pod \"9d3ce8b9-850f-48d5-ad02-87758cf50b24\" (UID: \"9d3ce8b9-850f-48d5-ad02-87758cf50b24\") "
Aug 04 00:16:38.376368 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.376370    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7055a49-ebe2-4868-9d17-f949a51ae86f-openshift-service-ca\") pod \"c7055a49-ebe2-4868-9d17-f949a51ae86f\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") "
Aug 04 00:16:38.376627 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.376425    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7055a49-ebe2-4868-9d17-f949a51ae86f-maas-api-tls\") pod \"c7055a49-ebe2-4868-9d17-f949a51ae86f\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") "
Aug 04 00:16:38.376627 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.376467    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gbpf\" (UniqueName: \"kubernetes.io/projected/c7055a49-ebe2-4868-9d17-f949a51ae86f-kube-api-access-2gbpf\") pod \"c7055a49-ebe2-4868-9d17-f949a51ae86f\" (UID: \"c7055a49-ebe2-4868-9d17-f949a51ae86f\") "
Aug 04 00:16:38.376627 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.376493    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f5f4966-0712-4f40-b5a1-365e889e0dd6-plugins-config-volume\") pod \"5f5f4966-0712-4f40-b5a1-365e889e0dd6\" (UID: \"5f5f4966-0712-4f40-b5a1-365e889e0dd6\") "
Aug 04 00:16:38.376627 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.376529    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtzs5\" (UniqueName: \"kubernetes.io/projected/5f5f4966-0712-4f40-b5a1-365e889e0dd6-kube-api-access-wtzs5\") pod \"5f5f4966-0712-4f40-b5a1-365e889e0dd6\" (UID: \"5f5f4966-0712-4f40-b5a1-365e889e0dd6\") "
Aug 04 00:16:38.376627 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.376605    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d3ce8b9-850f-48d5-ad02-87758cf50b24-plugins-config-volume\") pod \"9d3ce8b9-850f-48d5-ad02-87758cf50b24\" (UID: \"9d3ce8b9-850f-48d5-ad02-87758cf50b24\") "
Aug 04 00:16:38.376933 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.376841    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7055a49-ebe2-4868-9d17-f949a51ae86f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c7055a49-ebe2-4868-9d17-f949a51ae86f" (UID: "c7055a49-ebe2-4868-9d17-f949a51ae86f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:16:38.376933 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.376910    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f5f4966-0712-4f40-b5a1-365e889e0dd6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f5f4966-0712-4f40-b5a1-365e889e0dd6" (UID: "5f5f4966-0712-4f40-b5a1-365e889e0dd6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:16:38.377166 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.377138    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7055a49-ebe2-4868-9d17-f949a51ae86f-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:38.377296 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.377169    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f5f4966-0712-4f40-b5a1-365e889e0dd6-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:38.377296 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.377178    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d3ce8b9-850f-48d5-ad02-87758cf50b24-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d3ce8b9-850f-48d5-ad02-87758cf50b24" (UID: "9d3ce8b9-850f-48d5-ad02-87758cf50b24"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:16:38.378829 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.378799    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7055a49-ebe2-4868-9d17-f949a51ae86f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c7055a49-ebe2-4868-9d17-f949a51ae86f" (UID: "c7055a49-ebe2-4868-9d17-f949a51ae86f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:16:38.378829 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.378818    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7055a49-ebe2-4868-9d17-f949a51ae86f-kube-api-access-2gbpf" (OuterVolumeSpecName: "kube-api-access-2gbpf") pod "c7055a49-ebe2-4868-9d17-f949a51ae86f" (UID: "c7055a49-ebe2-4868-9d17-f949a51ae86f"). InnerVolumeSpecName "kube-api-access-2gbpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:16:38.378964 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.378854    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d3ce8b9-850f-48d5-ad02-87758cf50b24-kube-api-access-txgwz" (OuterVolumeSpecName: "kube-api-access-txgwz") pod "9d3ce8b9-850f-48d5-ad02-87758cf50b24" (UID: "9d3ce8b9-850f-48d5-ad02-87758cf50b24"). InnerVolumeSpecName "kube-api-access-txgwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:16:38.378964 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.378870    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f5f4966-0712-4f40-b5a1-365e889e0dd6-kube-api-access-wtzs5" (OuterVolumeSpecName: "kube-api-access-wtzs5") pod "5f5f4966-0712-4f40-b5a1-365e889e0dd6" (UID: "5f5f4966-0712-4f40-b5a1-365e889e0dd6"). InnerVolumeSpecName "kube-api-access-wtzs5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:16:38.448838 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.448807    2553 generic.go:356] "Generic (PLEG): container finished" podID="5f5f4966-0712-4f40-b5a1-365e889e0dd6" containerID="4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622" exitCode=0
Aug 04 00:16:38.448998 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.448854    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"
Aug 04 00:16:38.448998 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.448898    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92" event={"ID":"5f5f4966-0712-4f40-b5a1-365e889e0dd6","Type":"ContainerDied","Data":"4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622"}
Aug 04 00:16:38.448998 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.448950    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92" event={"ID":"5f5f4966-0712-4f40-b5a1-365e889e0dd6","Type":"ContainerDied","Data":"907d7aaa23465ec328dfe611645b3b624a66bc35f1aa5907196a09b43de81a30"}
Aug 04 00:16:38.448998 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.448974    2553 scope.go:117] "RemoveContainer" containerID="4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622"
Aug 04 00:16:38.450230 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.450196    2553 generic.go:356] "Generic (PLEG): container finished" podID="c7055a49-ebe2-4868-9d17-f949a51ae86f" containerID="00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2" exitCode=0
Aug 04 00:16:38.450335 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.450241    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"
Aug 04 00:16:38.450408 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.450237    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64" event={"ID":"c7055a49-ebe2-4868-9d17-f949a51ae86f","Type":"ContainerDied","Data":"00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2"}
Aug 04 00:16:38.450408 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.450398    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64" event={"ID":"c7055a49-ebe2-4868-9d17-f949a51ae86f","Type":"ContainerDied","Data":"57312a3f325e7862bb44aa93d19dd539387b2ccafbe91e876bb859ba10ac69a8"}
Aug 04 00:16:38.451635 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.451615    2553 generic.go:356] "Generic (PLEG): container finished" podID="9d3ce8b9-850f-48d5-ad02-87758cf50b24" containerID="97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b" exitCode=0
Aug 04 00:16:38.451760 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.451643    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p" event={"ID":"9d3ce8b9-850f-48d5-ad02-87758cf50b24","Type":"ContainerDied","Data":"97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b"}
Aug 04 00:16:38.451760 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.451659    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"
Aug 04 00:16:38.451760 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.451691    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p" event={"ID":"9d3ce8b9-850f-48d5-ad02-87758cf50b24","Type":"ContainerDied","Data":"45b0131ee360be012f6d00a78eaef4c67c8205f63ec44e77481e572486faad07"}
Aug 04 00:16:38.459342 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.459172    2553 scope.go:117] "RemoveContainer" containerID="4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622"
Aug 04 00:16:38.459461 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:16:38.459439    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622\": container with ID starting with 4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622 not found: ID does not exist" containerID="4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622"
Aug 04 00:16:38.459531 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.459473    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622"} err="failed to get container status \"4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622\": rpc error: code = NotFound desc = could not find container \"4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622\": container with ID starting with 4852992ade87a5ab890f39701cf1bb7fc844649d8e0877927620c688f8e42622 not found: ID does not exist"
Aug 04 00:16:38.459531 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.459494    2553 scope.go:117] "RemoveContainer" containerID="00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2"
Aug 04 00:16:38.468542 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.468524    2553 scope.go:117] "RemoveContainer" containerID="00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2"
Aug 04 00:16:38.468837 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:16:38.468819    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2\": container with ID starting with 00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2 not found: ID does not exist" containerID="00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2"
Aug 04 00:16:38.468916 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.468843    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2"} err="failed to get container status \"00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2\": rpc error: code = NotFound desc = could not find container \"00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2\": container with ID starting with 00e07656a8177eb3d379027b736011509764e5bea4cfbdf7b276c086d6016cf2 not found: ID does not exist"
Aug 04 00:16:38.468916 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.468869    2553 scope.go:117] "RemoveContainer" containerID="97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b"
Aug 04 00:16:38.473178 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.473158    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"]
Aug 04 00:16:38.477941 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.477917    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0f5b6-79456bbd8-nwf92"]
Aug 04 00:16:38.478471 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.478447    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gbpf\" (UniqueName: \"kubernetes.io/projected/c7055a49-ebe2-4868-9d17-f949a51ae86f-kube-api-access-2gbpf\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:38.478616 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.478600    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtzs5\" (UniqueName: \"kubernetes.io/projected/5f5f4966-0712-4f40-b5a1-365e889e0dd6-kube-api-access-wtzs5\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:38.478723 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.478694    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d3ce8b9-850f-48d5-ad02-87758cf50b24-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:38.478824 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.478813    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txgwz\" (UniqueName: \"kubernetes.io/projected/9d3ce8b9-850f-48d5-ad02-87758cf50b24-kube-api-access-txgwz\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:38.478915 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.478904    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7055a49-ebe2-4868-9d17-f949a51ae86f-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:16:38.478987 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.478524    2553 scope.go:117] "RemoveContainer" containerID="97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b"
Aug 04 00:16:38.479427 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:16:38.479401    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b\": container with ID starting with 97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b not found: ID does not exist" containerID="97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b"
Aug 04 00:16:38.479552 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.479529    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b"} err="failed to get container status \"97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b\": rpc error: code = NotFound desc = could not find container \"97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b\": container with ID starting with 97260bc9f95ef39fd6afcb780b7f4e62e09e946556974e575b448ef714d2c50b not found: ID does not exist"
Aug 04 00:16:38.488225 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.488198    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"]
Aug 04 00:16:38.491895 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.491870    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0f5b6-697cdcd5d6-bt27p"]
Aug 04 00:16:38.502237 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.502208    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"]
Aug 04 00:16:38.508325 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.508297    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0f5b6-976d68b5d-xfv64"]
Aug 04 00:16:38.764870 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.764795    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f5f4966-0712-4f40-b5a1-365e889e0dd6" path="/var/lib/kubelet/pods/5f5f4966-0712-4f40-b5a1-365e889e0dd6/volumes"
Aug 04 00:16:38.765165 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.765151    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d3ce8b9-850f-48d5-ad02-87758cf50b24" path="/var/lib/kubelet/pods/9d3ce8b9-850f-48d5-ad02-87758cf50b24/volumes"
Aug 04 00:16:38.765484 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:38.765473    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7055a49-ebe2-4868-9d17-f949a51ae86f" path="/var/lib/kubelet/pods/c7055a49-ebe2-4868-9d17-f949a51ae86f/volumes"
Aug 04 00:16:49.255392 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255353    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"]
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255735    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e337fae8-20d1-41c8-bbf4-b027f03b289a" containerName="maas-api"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255747    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e337fae8-20d1-41c8-bbf4-b027f03b289a" containerName="maas-api"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255756    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7055a49-ebe2-4868-9d17-f949a51ae86f" containerName="maas-api"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255762    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7055a49-ebe2-4868-9d17-f949a51ae86f" containerName="maas-api"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255769    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c859e1c-ee84-4fed-8784-0e99b944b8f5" containerName="payload-pre-processing"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255775    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c859e1c-ee84-4fed-8784-0e99b944b8f5" containerName="payload-pre-processing"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255791    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7301fa45-6478-4861-b120-3b37dd3c9845" containerName="payload-processing"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255797    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="7301fa45-6478-4861-b120-3b37dd3c9845" containerName="payload-processing"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255803    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f5f4966-0712-4f40-b5a1-365e889e0dd6" containerName="payload-pre-processing"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255807    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f5f4966-0712-4f40-b5a1-365e889e0dd6" containerName="payload-pre-processing"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255825    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d3ce8b9-850f-48d5-ad02-87758cf50b24" containerName="payload-processing"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255830    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d3ce8b9-850f-48d5-ad02-87758cf50b24" containerName="payload-processing"
Aug 04 00:16:49.255886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255891    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="e337fae8-20d1-41c8-bbf4-b027f03b289a" containerName="maas-api"
Aug 04 00:16:49.256576 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255901    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d3ce8b9-850f-48d5-ad02-87758cf50b24" containerName="payload-processing"
Aug 04 00:16:49.256576 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255908    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c859e1c-ee84-4fed-8784-0e99b944b8f5" containerName="payload-pre-processing"
Aug 04 00:16:49.256576 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255914    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f5f4966-0712-4f40-b5a1-365e889e0dd6" containerName="payload-pre-processing"
Aug 04 00:16:49.256576 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255920    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7055a49-ebe2-4868-9d17-f949a51ae86f" containerName="maas-api"
Aug 04 00:16:49.256576 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.255927    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="7301fa45-6478-4861-b120-3b37dd3c9845" containerName="payload-processing"
Aug 04 00:16:49.260785 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.260766    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:49.263030 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.263010    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b3817c-serving-cert\""
Aug 04 00:16:49.268834 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.268807    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4l8h\" (UniqueName: \"kubernetes.io/projected/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-kube-api-access-v4l8h\") pod \"maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:49.268956 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.268939    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-maas-api-tls\") pod \"maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:49.269036 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.268976    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-openshift-service-ca\") pod \"maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:49.269859 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.269838    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"]
Aug 04 00:16:49.370175 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.370146    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4l8h\" (UniqueName: \"kubernetes.io/projected/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-kube-api-access-v4l8h\") pod \"maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:49.370345 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.370213    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-maas-api-tls\") pod \"maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:49.370345 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.370236    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-openshift-service-ca\") pod \"maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:49.370949 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.370885    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-openshift-service-ca\") pod \"maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:49.372832 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.372809    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-maas-api-tls\") pod \"maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:49.381387 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.381357    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4l8h\" (UniqueName: \"kubernetes.io/projected/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-kube-api-access-v4l8h\") pod \"maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:49.572258 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.572184    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:49.737443 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.737354    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"]
Aug 04 00:16:49.739619 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:16:49.739592    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8de808b_2cff_4b05_b09d_dcbdde8fc8c0.slice/crio-7c113865ada812607e96f53202a842b6853a58b852f1079f5cba608e5eaf4ab1 WatchSource:0}: Error finding container 7c113865ada812607e96f53202a842b6853a58b852f1079f5cba608e5eaf4ab1: Status 404 returned error can't find the container with id 7c113865ada812607e96f53202a842b6853a58b852f1079f5cba608e5eaf4ab1
Aug 04 00:16:49.741118 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.741097    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 00:16:49.948005 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.947980    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"]
Aug 04 00:16:49.952640 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.952623    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:16:49.955146 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.955126    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-b3817c-dockercfg-tc5l6\""
Aug 04 00:16:49.955242 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.955127    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-b3817c\""
Aug 04 00:16:49.959223 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.959203    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"]
Aug 04 00:16:49.975622 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.975597    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slhr7\" (UniqueName: \"kubernetes.io/projected/469949cc-319d-4ce7-bdf4-8102d30f91a1-kube-api-access-slhr7\") pod \"payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4\" (UID: \"469949cc-319d-4ce7-bdf4-8102d30f91a1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:16:49.975755 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.975638    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/469949cc-319d-4ce7-bdf4-8102d30f91a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4\" (UID: \"469949cc-319d-4ce7-bdf4-8102d30f91a1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:16:49.983203 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.983182    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"]
Aug 04 00:16:49.986606 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.986587    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:16:49.998078 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:49.998049    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"]
Aug 04 00:16:50.076562 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.076532    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml9bg\" (UniqueName: \"kubernetes.io/projected/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-kube-api-access-ml9bg\") pod \"payload-processing-e2e-api-a-b3817c-795ddcb-dskg4\" (UID: \"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:16:50.076795 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.076581    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slhr7\" (UniqueName: \"kubernetes.io/projected/469949cc-319d-4ce7-bdf4-8102d30f91a1-kube-api-access-slhr7\") pod \"payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4\" (UID: \"469949cc-319d-4ce7-bdf4-8102d30f91a1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:16:50.076795 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.076666    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/469949cc-319d-4ce7-bdf4-8102d30f91a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4\" (UID: \"469949cc-319d-4ce7-bdf4-8102d30f91a1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:16:50.076795 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.076742    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b3817c-795ddcb-dskg4\" (UID: \"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:16:50.077368 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.077349    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/469949cc-319d-4ce7-bdf4-8102d30f91a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4\" (UID: \"469949cc-319d-4ce7-bdf4-8102d30f91a1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:16:50.084889 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.084864    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slhr7\" (UniqueName: \"kubernetes.io/projected/469949cc-319d-4ce7-bdf4-8102d30f91a1-kube-api-access-slhr7\") pod \"payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4\" (UID: \"469949cc-319d-4ce7-bdf4-8102d30f91a1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:16:50.177492 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.177458    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml9bg\" (UniqueName: \"kubernetes.io/projected/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-kube-api-access-ml9bg\") pod \"payload-processing-e2e-api-a-b3817c-795ddcb-dskg4\" (UID: \"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:16:50.177700 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.177535    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b3817c-795ddcb-dskg4\" (UID: \"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:16:50.178248 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.178228    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b3817c-795ddcb-dskg4\" (UID: \"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:16:50.185992 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.185974    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml9bg\" (UniqueName: \"kubernetes.io/projected/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-kube-api-access-ml9bg\") pod \"payload-processing-e2e-api-a-b3817c-795ddcb-dskg4\" (UID: \"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:16:50.263762 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.263661    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:16:50.297340 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.297310    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:16:50.415797 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.415763    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"]
Aug 04 00:16:50.419272 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:16:50.419242    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469949cc_319d_4ce7_bdf4_8102d30f91a1.slice/crio-2a52bd39f75b05ff56e115cc03e3cf5753df067810397c9e29c3ff3c8cd6a380 WatchSource:0}: Error finding container 2a52bd39f75b05ff56e115cc03e3cf5753df067810397c9e29c3ff3c8cd6a380: Status 404 returned error can't find the container with id 2a52bd39f75b05ff56e115cc03e3cf5753df067810397c9e29c3ff3c8cd6a380
Aug 04 00:16:50.444838 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.444795    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"]
Aug 04 00:16:50.447057 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:16:50.447029    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e7e7c2_82ce_4f7a_a805_6183f8ff7c4e.slice/crio-81bd1e572ef0826ed6231b7934242fbe000474c45341d6dab38ca78897f00222 WatchSource:0}: Error finding container 81bd1e572ef0826ed6231b7934242fbe000474c45341d6dab38ca78897f00222: Status 404 returned error can't find the container with id 81bd1e572ef0826ed6231b7934242fbe000474c45341d6dab38ca78897f00222
Aug 04 00:16:50.515893 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.515829    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4" event={"ID":"469949cc-319d-4ce7-bdf4-8102d30f91a1","Type":"ContainerStarted","Data":"2a52bd39f75b05ff56e115cc03e3cf5753df067810397c9e29c3ff3c8cd6a380"}
Aug 04 00:16:50.517300 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.517260    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw" event={"ID":"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0","Type":"ContainerStarted","Data":"7c113865ada812607e96f53202a842b6853a58b852f1079f5cba608e5eaf4ab1"}
Aug 04 00:16:50.519111 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:50.519084    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4" event={"ID":"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e","Type":"ContainerStarted","Data":"81bd1e572ef0826ed6231b7934242fbe000474c45341d6dab38ca78897f00222"}
Aug 04 00:16:51.525060 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:51.525024    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4" event={"ID":"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e","Type":"ContainerStarted","Data":"e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5"}
Aug 04 00:16:51.525602 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:51.525094    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:16:51.526532 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:51.526509    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4" event={"ID":"469949cc-319d-4ce7-bdf4-8102d30f91a1","Type":"ContainerStarted","Data":"84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab"}
Aug 04 00:16:51.526649 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:51.526565    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:16:51.528170 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:51.528141    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw" event={"ID":"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0","Type":"ContainerStarted","Data":"c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90"}
Aug 04 00:16:51.528305 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:51.528283    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:16:51.545145 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:51.545103    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4" podStartSLOduration=2.545089745 podStartE2EDuration="2.545089745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:16:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:16:51.54332033 +0000 UTC m=+2819.245026995" watchObservedRunningTime="2026-08-04 00:16:51.545089745 +0000 UTC m=+2819.246796433"
Aug 04 00:16:51.567943 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:51.567894    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4" podStartSLOduration=2.567874882 podStartE2EDuration="2.567874882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:16:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:16:51.565068188 +0000 UTC m=+2819.266774851" watchObservedRunningTime="2026-08-04 00:16:51.567874882 +0000 UTC m=+2819.269581551"
Aug 04 00:16:51.583370 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:51.583329    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw" podStartSLOduration=1.797296115 podStartE2EDuration="2.58331646s" totalImagesPullingTime="786.020345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:16:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:16:51.581306736 +0000 UTC m=+2819.283013401" watchObservedRunningTime="2026-08-04 00:16:51.58331646 +0000 UTC m=+2819.285023123"
Aug 04 00:16:57.539317 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:16:57.539287    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:17:02.535788 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:02.535757    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:17:02.536242 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:02.535829    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:17:17.596615 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.596585    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"]
Aug 04 00:17:17.600701 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.600683    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:17.603043 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.603021    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-35e594-serving-cert\""
Aug 04 00:17:17.609571 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.609546    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"]
Aug 04 00:17:17.735820 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.735782    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmsjm\" (UniqueName: \"kubernetes.io/projected/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-kube-api-access-zmsjm\") pod \"maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:17.735820 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.735826    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-openshift-service-ca\") pod \"maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:17.736031 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.735945    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-maas-api-tls\") pod \"maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:17.837198 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.837171    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-maas-api-tls\") pod \"maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:17.837359 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.837226    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmsjm\" (UniqueName: \"kubernetes.io/projected/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-kube-api-access-zmsjm\") pod \"maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:17.837359 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.837260    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-openshift-service-ca\") pod \"maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:17.837359 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:17:17.837306    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-serving-cert: secret "maas-api-e2e-api-b-35e594-serving-cert" not found
Aug 04 00:17:17.837468 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:17:17.837362    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-maas-api-tls podName:15efc11b-4d1f-4f21-8dee-ca186f6d9f5d nodeName:}" failed. No retries permitted until 2026-08-04 00:17:18.337346854 +0000 UTC m=+2846.039053496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-maas-api-tls") pod "maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh" (UID: "15efc11b-4d1f-4f21-8dee-ca186f6d9f5d") : secret "maas-api-e2e-api-b-35e594-serving-cert" not found
Aug 04 00:17:17.837995 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.837978    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-openshift-service-ca\") pod \"maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:17.845973 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:17.845955    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmsjm\" (UniqueName: \"kubernetes.io/projected/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-kube-api-access-zmsjm\") pod \"maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:18.289396 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.289363    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"]
Aug 04 00:17:18.293162 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.293133    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:17:18.295527 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.295506    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-35e594-dockercfg-d2lv6\""
Aug 04 00:17:18.295657 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.295534    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-35e594\""
Aug 04 00:17:18.302416 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.302391    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"]
Aug 04 00:17:18.315239 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.315202    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"]
Aug 04 00:17:18.319311 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.319291    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:17:18.329532 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.329503    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"]
Aug 04 00:17:18.342330 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.342311    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-maas-api-tls\") pod \"maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:18.344918 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.344900    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-maas-api-tls\") pod \"maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:18.443090 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.443060    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps\" (UID: \"b7e1773f-d188-41ae-8c24-fca0f6df7f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:17:18.443205 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.443107    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rncns\" (UniqueName: \"kubernetes.io/projected/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-kube-api-access-rncns\") pod \"payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps\" (UID: \"b7e1773f-d188-41ae-8c24-fca0f6df7f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:17:18.443205 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.443130    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13c60ada-0920-43fb-b0c1-3c55fed164b5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9\" (UID: \"13c60ada-0920-43fb-b0c1-3c55fed164b5\") " pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:17:18.443205 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.443194    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nfrg\" (UniqueName: \"kubernetes.io/projected/13c60ada-0920-43fb-b0c1-3c55fed164b5-kube-api-access-6nfrg\") pod \"payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9\" (UID: \"13c60ada-0920-43fb-b0c1-3c55fed164b5\") " pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:17:18.513130 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.513105    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:18.544563 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.544507    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nfrg\" (UniqueName: \"kubernetes.io/projected/13c60ada-0920-43fb-b0c1-3c55fed164b5-kube-api-access-6nfrg\") pod \"payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9\" (UID: \"13c60ada-0920-43fb-b0c1-3c55fed164b5\") " pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:17:18.544688 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.544564    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps\" (UID: \"b7e1773f-d188-41ae-8c24-fca0f6df7f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:17:18.544688 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.544606    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rncns\" (UniqueName: \"kubernetes.io/projected/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-kube-api-access-rncns\") pod \"payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps\" (UID: \"b7e1773f-d188-41ae-8c24-fca0f6df7f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:17:18.544688 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.544639    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13c60ada-0920-43fb-b0c1-3c55fed164b5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9\" (UID: \"13c60ada-0920-43fb-b0c1-3c55fed164b5\") " pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:17:18.545471 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.545448    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13c60ada-0920-43fb-b0c1-3c55fed164b5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9\" (UID: \"13c60ada-0920-43fb-b0c1-3c55fed164b5\") " pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:17:18.545535 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.545448    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps\" (UID: \"b7e1773f-d188-41ae-8c24-fca0f6df7f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:17:18.552922 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.552897    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rncns\" (UniqueName: \"kubernetes.io/projected/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-kube-api-access-rncns\") pod \"payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps\" (UID: \"b7e1773f-d188-41ae-8c24-fca0f6df7f6d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:17:18.553070 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.553046    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nfrg\" (UniqueName: \"kubernetes.io/projected/13c60ada-0920-43fb-b0c1-3c55fed164b5-kube-api-access-6nfrg\") pod \"payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9\" (UID: \"13c60ada-0920-43fb-b0c1-3c55fed164b5\") " pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:17:18.604968 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.604912    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:17:18.630525 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.630469    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:17:18.654550 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.654517    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"]
Aug 04 00:17:18.655611 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:17:18.655582    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15efc11b_4d1f_4f21_8dee_ca186f6d9f5d.slice/crio-35b4fd1bc6a2e6855e24c871241607b2a0a874d9ae264c69f0267f382421d277 WatchSource:0}: Error finding container 35b4fd1bc6a2e6855e24c871241607b2a0a874d9ae264c69f0267f382421d277: Status 404 returned error can't find the container with id 35b4fd1bc6a2e6855e24c871241607b2a0a874d9ae264c69f0267f382421d277
Aug 04 00:17:18.667815 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.667758    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh" event={"ID":"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d","Type":"ContainerStarted","Data":"35b4fd1bc6a2e6855e24c871241607b2a0a874d9ae264c69f0267f382421d277"}
Aug 04 00:17:18.767111 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:17:18.767078    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e1773f_d188_41ae_8c24_fca0f6df7f6d.slice/crio-dfaf370a4280635b6bd436050775a7e601b59ac30a89ad937097a10a8cb4741c WatchSource:0}: Error finding container dfaf370a4280635b6bd436050775a7e601b59ac30a89ad937097a10a8cb4741c: Status 404 returned error can't find the container with id dfaf370a4280635b6bd436050775a7e601b59ac30a89ad937097a10a8cb4741c
Aug 04 00:17:18.768899 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.768877    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"]
Aug 04 00:17:18.785189 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:18.785140    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"]
Aug 04 00:17:18.787406 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:17:18.787383    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c60ada_0920_43fb_b0c1_3c55fed164b5.slice/crio-6265d681dac50905b770323b1484e143d4faf7267e2943b451d4899ab591e1f4 WatchSource:0}: Error finding container 6265d681dac50905b770323b1484e143d4faf7267e2943b451d4899ab591e1f4: Status 404 returned error can't find the container with id 6265d681dac50905b770323b1484e143d4faf7267e2943b451d4899ab591e1f4
Aug 04 00:17:19.675860 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:19.675754    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps" event={"ID":"b7e1773f-d188-41ae-8c24-fca0f6df7f6d","Type":"ContainerStarted","Data":"8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085"}
Aug 04 00:17:19.675860 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:19.675831    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:17:19.675860 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:19.675854    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps" event={"ID":"b7e1773f-d188-41ae-8c24-fca0f6df7f6d","Type":"ContainerStarted","Data":"dfaf370a4280635b6bd436050775a7e601b59ac30a89ad937097a10a8cb4741c"}
Aug 04 00:17:19.677415 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:19.677390    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9" event={"ID":"13c60ada-0920-43fb-b0c1-3c55fed164b5","Type":"ContainerStarted","Data":"3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd"}
Aug 04 00:17:19.677543 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:19.677444    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:17:19.677543 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:19.677470    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9" event={"ID":"13c60ada-0920-43fb-b0c1-3c55fed164b5","Type":"ContainerStarted","Data":"6265d681dac50905b770323b1484e143d4faf7267e2943b451d4899ab591e1f4"}
Aug 04 00:17:19.685569 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:19.685529    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh" event={"ID":"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d","Type":"ContainerStarted","Data":"c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f"}
Aug 04 00:17:19.685673 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:19.685614    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:19.698647 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:19.698609    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps" podStartSLOduration=1.6985973589999999 podStartE2EDuration="1.698597359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:17:19.696061109 +0000 UTC m=+2847.397767774" watchObservedRunningTime="2026-08-04 00:17:19.698597359 +0000 UTC m=+2847.400304022"
Aug 04 00:17:19.712533 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:19.712493    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh" podStartSLOduration=2.114297282 podStartE2EDuration="2.712480884s" totalImagesPullingTime="598.183602ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:17:19.71128742 +0000 UTC m=+2847.412994095" watchObservedRunningTime="2026-08-04 00:17:19.712480884 +0000 UTC m=+2847.414187546"
Aug 04 00:17:19.726421 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:19.726364    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9" podStartSLOduration=1.7263519600000001 podStartE2EDuration="1.72635196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:17:19.725016604 +0000 UTC m=+2847.426723269" watchObservedRunningTime="2026-08-04 00:17:19.72635196 +0000 UTC m=+2847.428058623"
Aug 04 00:17:25.696359 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:25.696329    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:17:30.692622 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:30.692593    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:17:30.693061 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:17:30.692672    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:18:04.883465 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:04.883435    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"]
Aug 04 00:18:04.884055 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:04.883644    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw" podUID="d8de808b-2cff-4b05-b09d-dcbdde8fc8c0" containerName="maas-api" containerID="cri-o://c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90" gracePeriod=30
Aug 04 00:18:04.952246 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:04.952211    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"]
Aug 04 00:18:04.952492 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:04.952445    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4" podUID="75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e" containerName="payload-processing" containerID="cri-o://e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5" gracePeriod=30
Aug 04 00:18:04.954933 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:04.954903    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"]
Aug 04 00:18:04.955147 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:04.955099    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4" podUID="469949cc-319d-4ce7-bdf4-8102d30f91a1" containerName="payload-pre-processing" containerID="cri-o://84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab" gracePeriod=30
Aug 04 00:18:05.224062 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.224035    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:18:05.246034 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.246006    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:18:05.254242 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.254224    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:18:05.381049 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381020    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4l8h\" (UniqueName: \"kubernetes.io/projected/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-kube-api-access-v4l8h\") pod \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") "
Aug 04 00:18:05.381241 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381065    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-plugins-config-volume\") pod \"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e\" (UID: \"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e\") "
Aug 04 00:18:05.381241 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381117    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/469949cc-319d-4ce7-bdf4-8102d30f91a1-plugins-config-volume\") pod \"469949cc-319d-4ce7-bdf4-8102d30f91a1\" (UID: \"469949cc-319d-4ce7-bdf4-8102d30f91a1\") "
Aug 04 00:18:05.381241 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381141    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-openshift-service-ca\") pod \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") "
Aug 04 00:18:05.381241 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381207    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-maas-api-tls\") pod \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\" (UID: \"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0\") "
Aug 04 00:18:05.381241 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381237    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slhr7\" (UniqueName: \"kubernetes.io/projected/469949cc-319d-4ce7-bdf4-8102d30f91a1-kube-api-access-slhr7\") pod \"469949cc-319d-4ce7-bdf4-8102d30f91a1\" (UID: \"469949cc-319d-4ce7-bdf4-8102d30f91a1\") "
Aug 04 00:18:05.381514 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381270    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml9bg\" (UniqueName: \"kubernetes.io/projected/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-kube-api-access-ml9bg\") pod \"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e\" (UID: \"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e\") "
Aug 04 00:18:05.381571 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381542    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/469949cc-319d-4ce7-bdf4-8102d30f91a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "469949cc-319d-4ce7-bdf4-8102d30f91a1" (UID: "469949cc-319d-4ce7-bdf4-8102d30f91a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:18:05.381623 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381557    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e" (UID: "75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:18:05.381623 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381590    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d8de808b-2cff-4b05-b09d-dcbdde8fc8c0" (UID: "d8de808b-2cff-4b05-b09d-dcbdde8fc8c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:18:05.381798 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381777    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:05.381886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381804    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/469949cc-319d-4ce7-bdf4-8102d30f91a1-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:05.381886 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.381819    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:05.383446 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.383412    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d8de808b-2cff-4b05-b09d-dcbdde8fc8c0" (UID: "d8de808b-2cff-4b05-b09d-dcbdde8fc8c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:18:05.383558 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.383483    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-kube-api-access-v4l8h" (OuterVolumeSpecName: "kube-api-access-v4l8h") pod "d8de808b-2cff-4b05-b09d-dcbdde8fc8c0" (UID: "d8de808b-2cff-4b05-b09d-dcbdde8fc8c0"). InnerVolumeSpecName "kube-api-access-v4l8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:18:05.383558 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.383521    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/469949cc-319d-4ce7-bdf4-8102d30f91a1-kube-api-access-slhr7" (OuterVolumeSpecName: "kube-api-access-slhr7") pod "469949cc-319d-4ce7-bdf4-8102d30f91a1" (UID: "469949cc-319d-4ce7-bdf4-8102d30f91a1"). InnerVolumeSpecName "kube-api-access-slhr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:18:05.383558 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.383539    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-kube-api-access-ml9bg" (OuterVolumeSpecName: "kube-api-access-ml9bg") pod "75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e" (UID: "75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e"). InnerVolumeSpecName "kube-api-access-ml9bg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:18:05.482539 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.482508    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml9bg\" (UniqueName: \"kubernetes.io/projected/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e-kube-api-access-ml9bg\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:05.482539 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.482535    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4l8h\" (UniqueName: \"kubernetes.io/projected/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-kube-api-access-v4l8h\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:05.482539 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.482546    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:05.482795 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.482555    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slhr7\" (UniqueName: \"kubernetes.io/projected/469949cc-319d-4ce7-bdf4-8102d30f91a1-kube-api-access-slhr7\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:05.917980 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.917947    2553 generic.go:356] "Generic (PLEG): container finished" podID="d8de808b-2cff-4b05-b09d-dcbdde8fc8c0" containerID="c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90" exitCode=0
Aug 04 00:18:05.918417 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.917992    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"
Aug 04 00:18:05.918417 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.918034    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw" event={"ID":"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0","Type":"ContainerDied","Data":"c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90"}
Aug 04 00:18:05.918417 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.918096    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw" event={"ID":"d8de808b-2cff-4b05-b09d-dcbdde8fc8c0","Type":"ContainerDied","Data":"7c113865ada812607e96f53202a842b6853a58b852f1079f5cba608e5eaf4ab1"}
Aug 04 00:18:05.918417 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.918129    2553 scope.go:117] "RemoveContainer" containerID="c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90"
Aug 04 00:18:05.919402 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.919358    2553 generic.go:356] "Generic (PLEG): container finished" podID="75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e" containerID="e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5" exitCode=0
Aug 04 00:18:05.919478 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.919398    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4" event={"ID":"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e","Type":"ContainerDied","Data":"e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5"}
Aug 04 00:18:05.919478 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.919412    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"
Aug 04 00:18:05.919478 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.919447    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4" event={"ID":"75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e","Type":"ContainerDied","Data":"81bd1e572ef0826ed6231b7934242fbe000474c45341d6dab38ca78897f00222"}
Aug 04 00:18:05.920849 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.920828    2553 generic.go:356] "Generic (PLEG): container finished" podID="469949cc-319d-4ce7-bdf4-8102d30f91a1" containerID="84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab" exitCode=0
Aug 04 00:18:05.920962 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.920855    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4" event={"ID":"469949cc-319d-4ce7-bdf4-8102d30f91a1","Type":"ContainerDied","Data":"84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab"}
Aug 04 00:18:05.920962 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.920863    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"
Aug 04 00:18:05.920962 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.920889    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4" event={"ID":"469949cc-319d-4ce7-bdf4-8102d30f91a1","Type":"ContainerDied","Data":"2a52bd39f75b05ff56e115cc03e3cf5753df067810397c9e29c3ff3c8cd6a380"}
Aug 04 00:18:05.928525 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.928489    2553 scope.go:117] "RemoveContainer" containerID="c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90"
Aug 04 00:18:05.928824 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:18:05.928800    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90\": container with ID starting with c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90 not found: ID does not exist" containerID="c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90"
Aug 04 00:18:05.928937 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.928833    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90"} err="failed to get container status \"c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90\": rpc error: code = NotFound desc = could not find container \"c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90\": container with ID starting with c5229ea30f96a8fbb3e75be78a7c09c1a971b41396c8c4d84f0a39cab4e48a90 not found: ID does not exist"
Aug 04 00:18:05.928937 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.928856    2553 scope.go:117] "RemoveContainer" containerID="e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5"
Aug 04 00:18:05.937327 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.937313    2553 scope.go:117] "RemoveContainer" containerID="e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5"
Aug 04 00:18:05.937581 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:18:05.937562    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5\": container with ID starting with e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5 not found: ID does not exist" containerID="e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5"
Aug 04 00:18:05.937624 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.937587    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5"} err="failed to get container status \"e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5\": rpc error: code = NotFound desc = could not find container \"e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5\": container with ID starting with e9c4c9d4cca0d6d1b75d4f1819fbc13346fe360878c1f5609ea1fc8073d6e7f5 not found: ID does not exist"
Aug 04 00:18:05.937624 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.937603    2553 scope.go:117] "RemoveContainer" containerID="84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab"
Aug 04 00:18:05.947621 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.947599    2553 scope.go:117] "RemoveContainer" containerID="84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab"
Aug 04 00:18:05.948180 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:18:05.948160    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab\": container with ID starting with 84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab not found: ID does not exist" containerID="84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab"
Aug 04 00:18:05.948276 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.948190    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab"} err="failed to get container status \"84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab\": rpc error: code = NotFound desc = could not find container \"84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab\": container with ID starting with 84e859fb1ed17570d89a40897b95f88c8efe85b8a70bf97e99c5ee4606d72bab not found: ID does not exist"
Aug 04 00:18:05.948696 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.948677    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"]
Aug 04 00:18:05.954768 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.954746    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b3817c-7cd6459d5c-fxpfw"]
Aug 04 00:18:05.967729 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.967698    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"]
Aug 04 00:18:05.975976 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.975958    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b3817c-bfff7464d-z9ng4"]
Aug 04 00:18:05.987287 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.987268    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"]
Aug 04 00:18:05.991148 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:05.991130    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b3817c-795ddcb-dskg4"]
Aug 04 00:18:06.765550 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:06.765518    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="469949cc-319d-4ce7-bdf4-8102d30f91a1" path="/var/lib/kubelet/pods/469949cc-319d-4ce7-bdf4-8102d30f91a1/volumes"
Aug 04 00:18:06.765918 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:06.765903    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e" path="/var/lib/kubelet/pods/75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e/volumes"
Aug 04 00:18:06.766256 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:06.766243    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8de808b-2cff-4b05-b09d-dcbdde8fc8c0" path="/var/lib/kubelet/pods/d8de808b-2cff-4b05-b09d-dcbdde8fc8c0/volumes"
Aug 04 00:18:27.621780 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:27.621744    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"]
Aug 04 00:18:27.622233 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:27.622072    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh" podUID="15efc11b-4d1f-4f21-8dee-ca186f6d9f5d" containerName="maas-api" containerID="cri-o://c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f" gracePeriod=30
Aug 04 00:18:27.708249 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:27.708211    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"]
Aug 04 00:18:27.708514 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:27.708465    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9" podUID="13c60ada-0920-43fb-b0c1-3c55fed164b5" containerName="payload-processing" containerID="cri-o://3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd" gracePeriod=30
Aug 04 00:18:27.719917 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:27.719890    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"]
Aug 04 00:18:27.720250 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:27.720208    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps" podUID="b7e1773f-d188-41ae-8c24-fca0f6df7f6d" containerName="payload-pre-processing" containerID="cri-o://8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085" gracePeriod=30
Aug 04 00:18:27.990358 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:27.990332    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:18:28.007173 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.007151    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:18:28.013266 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.013245    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:18:28.046108 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.046055    2553 generic.go:356] "Generic (PLEG): container finished" podID="b7e1773f-d188-41ae-8c24-fca0f6df7f6d" containerID="8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085" exitCode=0
Aug 04 00:18:28.046233 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.046202    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"
Aug 04 00:18:28.046461 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.046445    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps" event={"ID":"b7e1773f-d188-41ae-8c24-fca0f6df7f6d","Type":"ContainerDied","Data":"8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085"}
Aug 04 00:18:28.046513 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.046487    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps" event={"ID":"b7e1773f-d188-41ae-8c24-fca0f6df7f6d","Type":"ContainerDied","Data":"dfaf370a4280635b6bd436050775a7e601b59ac30a89ad937097a10a8cb4741c"}
Aug 04 00:18:28.046572 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.046517    2553 scope.go:117] "RemoveContainer" containerID="8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085"
Aug 04 00:18:28.048271 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.048254    2553 generic.go:356] "Generic (PLEG): container finished" podID="13c60ada-0920-43fb-b0c1-3c55fed164b5" containerID="3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd" exitCode=0
Aug 04 00:18:28.048343 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.048312    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"
Aug 04 00:18:28.048407 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.048343    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9" event={"ID":"13c60ada-0920-43fb-b0c1-3c55fed164b5","Type":"ContainerDied","Data":"3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd"}
Aug 04 00:18:28.048461 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.048406    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9" event={"ID":"13c60ada-0920-43fb-b0c1-3c55fed164b5","Type":"ContainerDied","Data":"6265d681dac50905b770323b1484e143d4faf7267e2943b451d4899ab591e1f4"}
Aug 04 00:18:28.050215 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.050197    2553 generic.go:356] "Generic (PLEG): container finished" podID="15efc11b-4d1f-4f21-8dee-ca186f6d9f5d" containerID="c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f" exitCode=0
Aug 04 00:18:28.050322 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.050222    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh" event={"ID":"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d","Type":"ContainerDied","Data":"c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f"}
Aug 04 00:18:28.050322 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.050274    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh" event={"ID":"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d","Type":"ContainerDied","Data":"35b4fd1bc6a2e6855e24c871241607b2a0a874d9ae264c69f0267f382421d277"}
Aug 04 00:18:28.050322 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.050236    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"
Aug 04 00:18:28.057071 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.057051    2553 scope.go:117] "RemoveContainer" containerID="8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085"
Aug 04 00:18:28.057319 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:18:28.057303    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085\": container with ID starting with 8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085 not found: ID does not exist" containerID="8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085"
Aug 04 00:18:28.057377 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.057332    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085"} err="failed to get container status \"8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085\": rpc error: code = NotFound desc = could not find container \"8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085\": container with ID starting with 8c428a80ea6c71aea3d026ff5631932230abc1e61e0f74102b12ca5a6412f085 not found: ID does not exist"
Aug 04 00:18:28.057377 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.057347    2553 scope.go:117] "RemoveContainer" containerID="3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd"
Aug 04 00:18:28.065719 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.065687    2553 scope.go:117] "RemoveContainer" containerID="3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd"
Aug 04 00:18:28.066000 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:18:28.065978    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd\": container with ID starting with 3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd not found: ID does not exist" containerID="3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd"
Aug 04 00:18:28.066053 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.066013    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd"} err="failed to get container status \"3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd\": rpc error: code = NotFound desc = could not find container \"3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd\": container with ID starting with 3e2a3a47b76a577c3c291849384a26383594c851d14e94f467c7a04aaf7ebbbd not found: ID does not exist"
Aug 04 00:18:28.066053 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.066037    2553 scope.go:117] "RemoveContainer" containerID="c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f"
Aug 04 00:18:28.074586 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.074572    2553 scope.go:117] "RemoveContainer" containerID="c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f"
Aug 04 00:18:28.074823 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:18:28.074806    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f\": container with ID starting with c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f not found: ID does not exist" containerID="c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f"
Aug 04 00:18:28.074884 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.074830    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f"} err="failed to get container status \"c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f\": rpc error: code = NotFound desc = could not find container \"c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f\": container with ID starting with c7ac0cf7c33f04526cefc178ffa56dc2a44b79dcb5e2a25e380a755c9416e12f not found: ID does not exist"
Aug 04 00:18:28.082107 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082086    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-maas-api-tls\") pod \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") "
Aug 04 00:18:28.082208 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082128    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-plugins-config-volume\") pod \"b7e1773f-d188-41ae-8c24-fca0f6df7f6d\" (UID: \"b7e1773f-d188-41ae-8c24-fca0f6df7f6d\") "
Aug 04 00:18:28.082263 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082209    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmsjm\" (UniqueName: \"kubernetes.io/projected/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-kube-api-access-zmsjm\") pod \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") "
Aug 04 00:18:28.082316 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082277    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-openshift-service-ca\") pod \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\" (UID: \"15efc11b-4d1f-4f21-8dee-ca186f6d9f5d\") "
Aug 04 00:18:28.082384 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082365    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rncns\" (UniqueName: \"kubernetes.io/projected/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-kube-api-access-rncns\") pod \"b7e1773f-d188-41ae-8c24-fca0f6df7f6d\" (UID: \"b7e1773f-d188-41ae-8c24-fca0f6df7f6d\") "
Aug 04 00:18:28.082441 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082403    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13c60ada-0920-43fb-b0c1-3c55fed164b5-plugins-config-volume\") pod \"13c60ada-0920-43fb-b0c1-3c55fed164b5\" (UID: \"13c60ada-0920-43fb-b0c1-3c55fed164b5\") "
Aug 04 00:18:28.082497 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082440    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nfrg\" (UniqueName: \"kubernetes.io/projected/13c60ada-0920-43fb-b0c1-3c55fed164b5-kube-api-access-6nfrg\") pod \"13c60ada-0920-43fb-b0c1-3c55fed164b5\" (UID: \"13c60ada-0920-43fb-b0c1-3c55fed164b5\") "
Aug 04 00:18:28.082594 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082520    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b7e1773f-d188-41ae-8c24-fca0f6df7f6d" (UID: "b7e1773f-d188-41ae-8c24-fca0f6df7f6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:18:28.082594 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082562    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "15efc11b-4d1f-4f21-8dee-ca186f6d9f5d" (UID: "15efc11b-4d1f-4f21-8dee-ca186f6d9f5d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:18:28.082870 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082849    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13c60ada-0920-43fb-b0c1-3c55fed164b5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "13c60ada-0920-43fb-b0c1-3c55fed164b5" (UID: "13c60ada-0920-43fb-b0c1-3c55fed164b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:18:28.082964 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082949    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13c60ada-0920-43fb-b0c1-3c55fed164b5-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:28.083011 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082967    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:28.083011 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.082977    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:28.084313 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.084287    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "15efc11b-4d1f-4f21-8dee-ca186f6d9f5d" (UID: "15efc11b-4d1f-4f21-8dee-ca186f6d9f5d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:18:28.084414 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.084348    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-kube-api-access-zmsjm" (OuterVolumeSpecName: "kube-api-access-zmsjm") pod "15efc11b-4d1f-4f21-8dee-ca186f6d9f5d" (UID: "15efc11b-4d1f-4f21-8dee-ca186f6d9f5d"). InnerVolumeSpecName "kube-api-access-zmsjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:18:28.084735 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.084698    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-kube-api-access-rncns" (OuterVolumeSpecName: "kube-api-access-rncns") pod "b7e1773f-d188-41ae-8c24-fca0f6df7f6d" (UID: "b7e1773f-d188-41ae-8c24-fca0f6df7f6d"). InnerVolumeSpecName "kube-api-access-rncns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:18:28.084992 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.084973    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13c60ada-0920-43fb-b0c1-3c55fed164b5-kube-api-access-6nfrg" (OuterVolumeSpecName: "kube-api-access-6nfrg") pod "13c60ada-0920-43fb-b0c1-3c55fed164b5" (UID: "13c60ada-0920-43fb-b0c1-3c55fed164b5"). InnerVolumeSpecName "kube-api-access-6nfrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:18:28.184394 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.184372    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rncns\" (UniqueName: \"kubernetes.io/projected/b7e1773f-d188-41ae-8c24-fca0f6df7f6d-kube-api-access-rncns\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:28.184394 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.184394    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nfrg\" (UniqueName: \"kubernetes.io/projected/13c60ada-0920-43fb-b0c1-3c55fed164b5-kube-api-access-6nfrg\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:28.184515 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.184403    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:28.184515 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.184413    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmsjm\" (UniqueName: \"kubernetes.io/projected/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d-kube-api-access-zmsjm\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:18:28.374908 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.374879    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"]
Aug 04 00:18:28.378688 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.378665    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-35e594-69c4df8bbd-qrmps"]
Aug 04 00:18:28.391188 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.391160    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"]
Aug 04 00:18:28.397537 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.397514    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-35e594-65dbc76cb-8bdk9"]
Aug 04 00:18:28.408758 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.408737    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"]
Aug 04 00:18:28.413087 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.413069    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-35e594-58c4968dd7-jj4qh"]
Aug 04 00:18:28.764272 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.764235    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13c60ada-0920-43fb-b0c1-3c55fed164b5" path="/var/lib/kubelet/pods/13c60ada-0920-43fb-b0c1-3c55fed164b5/volumes"
Aug 04 00:18:28.764663 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.764649    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15efc11b-4d1f-4f21-8dee-ca186f6d9f5d" path="/var/lib/kubelet/pods/15efc11b-4d1f-4f21-8dee-ca186f6d9f5d/volumes"
Aug 04 00:18:28.765002 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:28.764989    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7e1773f-d188-41ae-8c24-fca0f6df7f6d" path="/var/lib/kubelet/pods/b7e1773f-d188-41ae-8c24-fca0f6df7f6d/volumes"
Aug 04 00:18:39.817897 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.817864    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"]
Aug 04 00:18:39.818362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818274    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8de808b-2cff-4b05-b09d-dcbdde8fc8c0" containerName="maas-api"
Aug 04 00:18:39.818362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818287    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8de808b-2cff-4b05-b09d-dcbdde8fc8c0" containerName="maas-api"
Aug 04 00:18:39.818362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818298    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="469949cc-319d-4ce7-bdf4-8102d30f91a1" containerName="payload-pre-processing"
Aug 04 00:18:39.818362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818304    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="469949cc-319d-4ce7-bdf4-8102d30f91a1" containerName="payload-pre-processing"
Aug 04 00:18:39.818362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818317    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15efc11b-4d1f-4f21-8dee-ca186f6d9f5d" containerName="maas-api"
Aug 04 00:18:39.818362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818323    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="15efc11b-4d1f-4f21-8dee-ca186f6d9f5d" containerName="maas-api"
Aug 04 00:18:39.818362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818331    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13c60ada-0920-43fb-b0c1-3c55fed164b5" containerName="payload-processing"
Aug 04 00:18:39.818362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818340    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c60ada-0920-43fb-b0c1-3c55fed164b5" containerName="payload-processing"
Aug 04 00:18:39.818362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818354    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e" containerName="payload-processing"
Aug 04 00:18:39.818362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818359    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e" containerName="payload-processing"
Aug 04 00:18:39.818362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818364    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e1773f-d188-41ae-8c24-fca0f6df7f6d" containerName="payload-pre-processing"
Aug 04 00:18:39.818696 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818369    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e1773f-d188-41ae-8c24-fca0f6df7f6d" containerName="payload-pre-processing"
Aug 04 00:18:39.818696 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818441    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e1773f-d188-41ae-8c24-fca0f6df7f6d" containerName="payload-pre-processing"
Aug 04 00:18:39.818696 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818450    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8de808b-2cff-4b05-b09d-dcbdde8fc8c0" containerName="maas-api"
Aug 04 00:18:39.818696 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818457    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e7e7c2-82ce-4f7a-a805-6183f8ff7c4e" containerName="payload-processing"
Aug 04 00:18:39.818696 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818462    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="13c60ada-0920-43fb-b0c1-3c55fed164b5" containerName="payload-processing"
Aug 04 00:18:39.818696 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818467    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="15efc11b-4d1f-4f21-8dee-ca186f6d9f5d" containerName="maas-api"
Aug 04 00:18:39.818696 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.818473    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="469949cc-319d-4ce7-bdf4-8102d30f91a1" containerName="payload-pre-processing"
Aug 04 00:18:39.821660 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.821645    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:39.824308 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.824289    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-1769a3-serving-cert\""
Aug 04 00:18:39.832852 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.832829    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"]
Aug 04 00:18:39.884041 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.884009    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1047395c-83fc-49f4-839f-3df342b461e2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:39.884208 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.884138    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mg7b\" (UniqueName: \"kubernetes.io/projected/1047395c-83fc-49f4-839f-3df342b461e2-kube-api-access-7mg7b\") pod \"maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:39.884208 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.884194    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1047395c-83fc-49f4-839f-3df342b461e2-maas-api-tls\") pod \"maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:39.985809 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.985772    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mg7b\" (UniqueName: \"kubernetes.io/projected/1047395c-83fc-49f4-839f-3df342b461e2-kube-api-access-7mg7b\") pod \"maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:39.986012 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.985827    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1047395c-83fc-49f4-839f-3df342b461e2-maas-api-tls\") pod \"maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:39.986012 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.985897    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1047395c-83fc-49f4-839f-3df342b461e2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:39.986133 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:18:39.986012    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-serving-cert: secret "maas-api-e2e-shared-a-1769a3-serving-cert" not found
Aug 04 00:18:39.986133 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:18:39.986104    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1047395c-83fc-49f4-839f-3df342b461e2-maas-api-tls podName:1047395c-83fc-49f4-839f-3df342b461e2 nodeName:}" failed. No retries permitted until 2026-08-04 00:18:40.486081374 +0000 UTC m=+2928.187788035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1047395c-83fc-49f4-839f-3df342b461e2-maas-api-tls") pod "maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb" (UID: "1047395c-83fc-49f4-839f-3df342b461e2") : secret "maas-api-e2e-shared-a-1769a3-serving-cert" not found
Aug 04 00:18:39.986468 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.986452    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1047395c-83fc-49f4-839f-3df342b461e2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:39.994844 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:39.994824    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mg7b\" (UniqueName: \"kubernetes.io/projected/1047395c-83fc-49f4-839f-3df342b461e2-kube-api-access-7mg7b\") pod \"maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:40.463135 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.463099    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"]
Aug 04 00:18:40.466536 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.466507    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:18:40.468923 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.468899    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-1769a3\""
Aug 04 00:18:40.469026 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.468974    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-1769a3-dockercfg-qfz7v\""
Aug 04 00:18:40.478204 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.478180    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"]
Aug 04 00:18:40.491874 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.491838    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1047395c-83fc-49f4-839f-3df342b461e2-maas-api-tls\") pod \"maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:40.494555 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.494528    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1047395c-83fc-49f4-839f-3df342b461e2-maas-api-tls\") pod \"maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:40.505642 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.505616    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"]
Aug 04 00:18:40.509508 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.509491    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:18:40.518536 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.518428    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"]
Aug 04 00:18:40.593464 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.593420    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f3a85b9-5936-430b-ac98-c0c7499232b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42\" (UID: \"8f3a85b9-5936-430b-ac98-c0c7499232b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:18:40.593639 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.593492    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m86h\" (UniqueName: \"kubernetes.io/projected/8f3a85b9-5936-430b-ac98-c0c7499232b5-kube-api-access-6m86h\") pod \"payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42\" (UID: \"8f3a85b9-5936-430b-ac98-c0c7499232b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:18:40.593639 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.593597    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4fmh\" (UniqueName: \"kubernetes.io/projected/95a55fb2-cc4f-4404-9a64-8fa13af10d06-kube-api-access-b4fmh\") pod \"payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz\" (UID: \"95a55fb2-cc4f-4404-9a64-8fa13af10d06\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:18:40.593826 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.593664    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95a55fb2-cc4f-4404-9a64-8fa13af10d06-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz\" (UID: \"95a55fb2-cc4f-4404-9a64-8fa13af10d06\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:18:40.694436 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.694397    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m86h\" (UniqueName: \"kubernetes.io/projected/8f3a85b9-5936-430b-ac98-c0c7499232b5-kube-api-access-6m86h\") pod \"payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42\" (UID: \"8f3a85b9-5936-430b-ac98-c0c7499232b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:18:40.694645 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.694479    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4fmh\" (UniqueName: \"kubernetes.io/projected/95a55fb2-cc4f-4404-9a64-8fa13af10d06-kube-api-access-b4fmh\") pod \"payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz\" (UID: \"95a55fb2-cc4f-4404-9a64-8fa13af10d06\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:18:40.694645 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.694529    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95a55fb2-cc4f-4404-9a64-8fa13af10d06-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz\" (UID: \"95a55fb2-cc4f-4404-9a64-8fa13af10d06\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:18:40.694645 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.694599    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f3a85b9-5936-430b-ac98-c0c7499232b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42\" (UID: \"8f3a85b9-5936-430b-ac98-c0c7499232b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:18:40.695328 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.695303    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95a55fb2-cc4f-4404-9a64-8fa13af10d06-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz\" (UID: \"95a55fb2-cc4f-4404-9a64-8fa13af10d06\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:18:40.695417 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.695343    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f3a85b9-5936-430b-ac98-c0c7499232b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42\" (UID: \"8f3a85b9-5936-430b-ac98-c0c7499232b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:18:40.704973 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.704937    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4fmh\" (UniqueName: \"kubernetes.io/projected/95a55fb2-cc4f-4404-9a64-8fa13af10d06-kube-api-access-b4fmh\") pod \"payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz\" (UID: \"95a55fb2-cc4f-4404-9a64-8fa13af10d06\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:18:40.704973 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.704965    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m86h\" (UniqueName: \"kubernetes.io/projected/8f3a85b9-5936-430b-ac98-c0c7499232b5-kube-api-access-6m86h\") pod \"payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42\" (UID: \"8f3a85b9-5936-430b-ac98-c0c7499232b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:18:40.733865 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.733773    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:40.777976 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.777944    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:18:40.822549 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.822518    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:18:40.895450 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.895246    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"]
Aug 04 00:18:40.989300 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:40.989276    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"]
Aug 04 00:18:40.990895 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:18:40.990870    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a55fb2_cc4f_4404_9a64_8fa13af10d06.slice/crio-f77302c3c2b62919be29707fd55b3e3a699007e97cb3a209f72bffbffd6a36fb WatchSource:0}: Error finding container f77302c3c2b62919be29707fd55b3e3a699007e97cb3a209f72bffbffd6a36fb: Status 404 returned error can't find the container with id f77302c3c2b62919be29707fd55b3e3a699007e97cb3a209f72bffbffd6a36fb
Aug 04 00:18:41.127075 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:41.127039    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb" event={"ID":"1047395c-83fc-49f4-839f-3df342b461e2","Type":"ContainerStarted","Data":"909adb6e8c49cb08a9977abc05f83c3ff66f1d944d5b46c24d6bf7146a0c9307"}
Aug 04 00:18:41.128727 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:41.128686    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz" event={"ID":"95a55fb2-cc4f-4404-9a64-8fa13af10d06","Type":"ContainerStarted","Data":"7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77"}
Aug 04 00:18:41.128875 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:41.128762    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz" event={"ID":"95a55fb2-cc4f-4404-9a64-8fa13af10d06","Type":"ContainerStarted","Data":"f77302c3c2b62919be29707fd55b3e3a699007e97cb3a209f72bffbffd6a36fb"}
Aug 04 00:18:41.128875 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:41.128827    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:18:41.148617 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:41.148555    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz" podStartSLOduration=1.1485335939999999 podStartE2EDuration="1.148533594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:18:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:18:41.147854481 +0000 UTC m=+2928.849561142" watchObservedRunningTime="2026-08-04 00:18:41.148533594 +0000 UTC m=+2928.850240259"
Aug 04 00:18:41.155071 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:41.155043    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"]
Aug 04 00:18:41.157478 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:18:41.157454    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f3a85b9_5936_430b_ac98_c0c7499232b5.slice/crio-78c6ed71c06c49d8c73faa02de5d3239eb01db0f91ee5daad1450c653c55a675 WatchSource:0}: Error finding container 78c6ed71c06c49d8c73faa02de5d3239eb01db0f91ee5daad1450c653c55a675: Status 404 returned error can't find the container with id 78c6ed71c06c49d8c73faa02de5d3239eb01db0f91ee5daad1450c653c55a675
Aug 04 00:18:42.138337 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:42.138300    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb" event={"ID":"1047395c-83fc-49f4-839f-3df342b461e2","Type":"ContainerStarted","Data":"f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6"}
Aug 04 00:18:42.138889 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:42.138868    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:42.140300 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:42.140275    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42" event={"ID":"8f3a85b9-5936-430b-ac98-c0c7499232b5","Type":"ContainerStarted","Data":"b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8"}
Aug 04 00:18:42.140421 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:42.140325    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42" event={"ID":"8f3a85b9-5936-430b-ac98-c0c7499232b5","Type":"ContainerStarted","Data":"78c6ed71c06c49d8c73faa02de5d3239eb01db0f91ee5daad1450c653c55a675"}
Aug 04 00:18:42.140421 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:42.140355    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:18:42.157113 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:42.157068    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb" podStartSLOduration=2.783888786 podStartE2EDuration="3.157054459s" totalImagesPullingTime="373.165673ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:18:42.154290364 +0000 UTC m=+2929.855997030" watchObservedRunningTime="2026-08-04 00:18:42.157054459 +0000 UTC m=+2929.858761123"
Aug 04 00:18:42.173413 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:42.173375    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42" podStartSLOduration=2.173362578 podStartE2EDuration="2.173362578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:18:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:18:42.172842575 +0000 UTC m=+2929.874549239" watchObservedRunningTime="2026-08-04 00:18:42.173362578 +0000 UTC m=+2929.875069242"
Aug 04 00:18:49.156323 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:49.156293    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:18:52.143546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:52.143513    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:18:53.148441 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:18:53.148415    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:19:08.015799 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.015767    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"]
Aug 04 00:19:08.019549 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.019528    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:08.021798 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.021772    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-98f978-serving-cert\""
Aug 04 00:19:08.030433 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.030409    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"]
Aug 04 00:19:08.057020 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.056937    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd833c55-3b2d-42d4-96b3-27d799bde070-maas-api-tls\") pod \"maas-api-e2e-shared-b-98f978-76657fc476-m8vgm\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:08.057223 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.057042    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd833c55-3b2d-42d4-96b3-27d799bde070-openshift-service-ca\") pod \"maas-api-e2e-shared-b-98f978-76657fc476-m8vgm\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:08.057223 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.057181    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrcs6\" (UniqueName: \"kubernetes.io/projected/cd833c55-3b2d-42d4-96b3-27d799bde070-kube-api-access-wrcs6\") pod \"maas-api-e2e-shared-b-98f978-76657fc476-m8vgm\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:08.158406 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.158370    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd833c55-3b2d-42d4-96b3-27d799bde070-openshift-service-ca\") pod \"maas-api-e2e-shared-b-98f978-76657fc476-m8vgm\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:08.158575 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.158452    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrcs6\" (UniqueName: \"kubernetes.io/projected/cd833c55-3b2d-42d4-96b3-27d799bde070-kube-api-access-wrcs6\") pod \"maas-api-e2e-shared-b-98f978-76657fc476-m8vgm\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:08.158575 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.158498    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd833c55-3b2d-42d4-96b3-27d799bde070-maas-api-tls\") pod \"maas-api-e2e-shared-b-98f978-76657fc476-m8vgm\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:08.158654 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:19:08.158594    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-serving-cert: secret "maas-api-e2e-shared-b-98f978-serving-cert" not found
Aug 04 00:19:08.158689 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:19:08.158662    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd833c55-3b2d-42d4-96b3-27d799bde070-maas-api-tls podName:cd833c55-3b2d-42d4-96b3-27d799bde070 nodeName:}" failed. No retries permitted until 2026-08-04 00:19:08.658644184 +0000 UTC m=+2956.360350828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cd833c55-3b2d-42d4-96b3-27d799bde070-maas-api-tls") pod "maas-api-e2e-shared-b-98f978-76657fc476-m8vgm" (UID: "cd833c55-3b2d-42d4-96b3-27d799bde070") : secret "maas-api-e2e-shared-b-98f978-serving-cert" not found
Aug 04 00:19:08.159037 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.159019    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd833c55-3b2d-42d4-96b3-27d799bde070-openshift-service-ca\") pod \"maas-api-e2e-shared-b-98f978-76657fc476-m8vgm\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:08.174526 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.174496    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrcs6\" (UniqueName: \"kubernetes.io/projected/cd833c55-3b2d-42d4-96b3-27d799bde070-kube-api-access-wrcs6\") pod \"maas-api-e2e-shared-b-98f978-76657fc476-m8vgm\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:08.663199 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.663161    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd833c55-3b2d-42d4-96b3-27d799bde070-maas-api-tls\") pod \"maas-api-e2e-shared-b-98f978-76657fc476-m8vgm\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:08.665818 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.665795    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd833c55-3b2d-42d4-96b3-27d799bde070-maas-api-tls\") pod \"maas-api-e2e-shared-b-98f978-76657fc476-m8vgm\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:08.704589 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.704541    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"]
Aug 04 00:19:08.708435 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.708412    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:19:08.710892 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.710871    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-98f978\""
Aug 04 00:19:08.725779 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.725758    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"]
Aug 04 00:19:08.729214 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.729187    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:19:08.731696 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.731672    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-98f978-dockercfg-qqg6d\""
Aug 04 00:19:08.732187 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.732166    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"]
Aug 04 00:19:08.738011 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.737992    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"]
Aug 04 00:19:08.764852 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.764739    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78ce4994-95b0-4893-8d14-9f8b0c49b416-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6\" (UID: \"78ce4994-95b0-4893-8d14-9f8b0c49b416\") " pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:19:08.764852 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.764790    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbxch\" (UniqueName: \"kubernetes.io/projected/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-kube-api-access-rbxch\") pod \"payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2\" (UID: \"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:19:08.764852 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.764850    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fbnm\" (UniqueName: \"kubernetes.io/projected/78ce4994-95b0-4893-8d14-9f8b0c49b416-kube-api-access-6fbnm\") pod \"payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6\" (UID: \"78ce4994-95b0-4893-8d14-9f8b0c49b416\") " pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:19:08.765108 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.764929    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2\" (UID: \"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:19:08.866123 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.866089    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2\" (UID: \"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:19:08.866276 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.866167    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78ce4994-95b0-4893-8d14-9f8b0c49b416-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6\" (UID: \"78ce4994-95b0-4893-8d14-9f8b0c49b416\") " pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:19:08.866276 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.866189    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbxch\" (UniqueName: \"kubernetes.io/projected/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-kube-api-access-rbxch\") pod \"payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2\" (UID: \"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:19:08.866276 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.866222    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fbnm\" (UniqueName: \"kubernetes.io/projected/78ce4994-95b0-4893-8d14-9f8b0c49b416-kube-api-access-6fbnm\") pod \"payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6\" (UID: \"78ce4994-95b0-4893-8d14-9f8b0c49b416\") " pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:19:08.866869 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.866850    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2\" (UID: \"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:19:08.866929 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.866913    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78ce4994-95b0-4893-8d14-9f8b0c49b416-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6\" (UID: \"78ce4994-95b0-4893-8d14-9f8b0c49b416\") " pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:19:08.877538 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.877512    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbxch\" (UniqueName: \"kubernetes.io/projected/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-kube-api-access-rbxch\") pod \"payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2\" (UID: \"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:19:08.877781 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.877754    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fbnm\" (UniqueName: \"kubernetes.io/projected/78ce4994-95b0-4893-8d14-9f8b0c49b416-kube-api-access-6fbnm\") pod \"payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6\" (UID: \"78ce4994-95b0-4893-8d14-9f8b0c49b416\") " pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:19:08.931288 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:08.931261    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:09.019379 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:09.019016    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:19:09.042323 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:09.042293    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:19:09.070658 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:09.070059    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"]
Aug 04 00:19:09.072970 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:19:09.072944    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd833c55_3b2d_42d4_96b3_27d799bde070.slice/crio-310bf257c70882b6b1669525cf4c911ba7d49769c59cb1a671a78337d5817b5e WatchSource:0}: Error finding container 310bf257c70882b6b1669525cf4c911ba7d49769c59cb1a671a78337d5817b5e: Status 404 returned error can't find the container with id 310bf257c70882b6b1669525cf4c911ba7d49769c59cb1a671a78337d5817b5e
Aug 04 00:19:09.180113 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:09.180038    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"]
Aug 04 00:19:09.183263 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:19:09.182928    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aa7e0b6_cb66_4f78_b9fc_2df4dee472d5.slice/crio-9cad16ffb4280de29f68cfec203e410c1ee890ce406f58802556b14af9cf7e1d WatchSource:0}: Error finding container 9cad16ffb4280de29f68cfec203e410c1ee890ce406f58802556b14af9cf7e1d: Status 404 returned error can't find the container with id 9cad16ffb4280de29f68cfec203e410c1ee890ce406f58802556b14af9cf7e1d
Aug 04 00:19:09.215400 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:09.215377    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"]
Aug 04 00:19:09.224914 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:19:09.224873    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ce4994_95b0_4893_8d14_9f8b0c49b416.slice/crio-e109f91878b5e3b9cf2a3ed36686e911b26588d11918727d6ec4175749222015 WatchSource:0}: Error finding container e109f91878b5e3b9cf2a3ed36686e911b26588d11918727d6ec4175749222015: Status 404 returned error can't find the container with id e109f91878b5e3b9cf2a3ed36686e911b26588d11918727d6ec4175749222015
Aug 04 00:19:09.283725 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:09.283678    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm" event={"ID":"cd833c55-3b2d-42d4-96b3-27d799bde070","Type":"ContainerStarted","Data":"310bf257c70882b6b1669525cf4c911ba7d49769c59cb1a671a78337d5817b5e"}
Aug 04 00:19:09.284996 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:09.284971    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6" event={"ID":"78ce4994-95b0-4893-8d14-9f8b0c49b416","Type":"ContainerStarted","Data":"e109f91878b5e3b9cf2a3ed36686e911b26588d11918727d6ec4175749222015"}
Aug 04 00:19:09.286669 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:09.286646    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2" event={"ID":"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5","Type":"ContainerStarted","Data":"dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b"}
Aug 04 00:19:09.286809 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:09.286689    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2" event={"ID":"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5","Type":"ContainerStarted","Data":"9cad16ffb4280de29f68cfec203e410c1ee890ce406f58802556b14af9cf7e1d"}
Aug 04 00:19:09.286809 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:09.286781    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:19:09.307677 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:09.307633    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2" podStartSLOduration=1.3076178760000001 podStartE2EDuration="1.307617876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:19:09.306970145 +0000 UTC m=+2957.008676809" watchObservedRunningTime="2026-08-04 00:19:09.307617876 +0000 UTC m=+2957.009324539"
Aug 04 00:19:10.293323 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:10.293282    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6" event={"ID":"78ce4994-95b0-4893-8d14-9f8b0c49b416","Type":"ContainerStarted","Data":"5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f"}
Aug 04 00:19:10.293847 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:10.293358    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:19:10.294852 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:10.294825    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm" event={"ID":"cd833c55-3b2d-42d4-96b3-27d799bde070","Type":"ContainerStarted","Data":"35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e"}
Aug 04 00:19:10.308039 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:10.307997    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6" podStartSLOduration=2.307981961 podStartE2EDuration="2.307981961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:19:10.306576893 +0000 UTC m=+2958.008283559" watchObservedRunningTime="2026-08-04 00:19:10.307981961 +0000 UTC m=+2958.009688603"
Aug 04 00:19:11.306078 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:11.306042    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:17.317783 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:17.317688    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:19:17.336288 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:17.336234    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm" podStartSLOduration=9.969662875000001 podStartE2EDuration="10.336216912s" totalImagesPullingTime="366.554037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:19:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:19:10.321252732 +0000 UTC m=+2958.022959397" watchObservedRunningTime="2026-08-04 00:19:17.336216912 +0000 UTC m=+2965.037923578"
Aug 04 00:19:20.297094 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:20.297064    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:19:21.307666 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:19:21.307634    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:20:53.099953 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:20:53.099922    2553 scope.go:117] "RemoveContainer" containerID="2335847873a6b1b584e5d99289bedf9db93fe519c708b7e3ff2678205887a0d5"
Aug 04 00:20:53.109113 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:20:53.109088    2553 scope.go:117] "RemoveContainer" containerID="abfa83f7502f92e5d8b1dd5b03d0eeae5ca9297505d1b0bad1b35b8660354ed5"
Aug 04 00:25:17.441997 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.441905    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"]
Aug 04 00:25:17.445541 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.445526    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.449175 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.449150    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1769a3\"/\"default-dockercfg-4jqnj\""
Aug 04 00:25:17.449316 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.449180    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1769a3\"/\"rate-test-model-1769a3-kserve-self-signed-certs\""
Aug 04 00:25:17.449316 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.449217    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1769a3\"/\"openshift-service-ca.crt\""
Aug 04 00:25:17.449316 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.449161    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1769a3\"/\"kube-root-ca.crt\""
Aug 04 00:25:17.455857 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.455837    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"]
Aug 04 00:25:17.514969 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.514937    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-tmp-dir\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.514969 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.514972    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8hpp\" (UniqueName: \"kubernetes.io/projected/3b91d063-9324-4ce4-bb7c-c2790c702632-kube-api-access-w8hpp\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.515164 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.515049    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b91d063-9324-4ce4-bb7c-c2790c702632-tls-certs\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.515164 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.515072    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-model-cache\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.515242 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.515158    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-home\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.515242 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.515227    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-dshm\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.616505 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.616474    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-tmp-dir\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.616505 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.616504    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8hpp\" (UniqueName: \"kubernetes.io/projected/3b91d063-9324-4ce4-bb7c-c2790c702632-kube-api-access-w8hpp\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.616739 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.616535    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b91d063-9324-4ce4-bb7c-c2790c702632-tls-certs\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.616739 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.616554    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-model-cache\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.616739 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.616576    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-home\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.616739 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.616605    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-dshm\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.616965 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.616939    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-tmp-dir\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.616965 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.616952    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-model-cache\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.617050 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.617012    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-home\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.618946 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.618931    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-dshm\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.619200 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.619185    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b91d063-9324-4ce4-bb7c-c2790c702632-tls-certs\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.624940 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.624919    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8hpp\" (UniqueName: \"kubernetes.io/projected/3b91d063-9324-4ce4-bb7c-c2790c702632-kube-api-access-w8hpp\") pod \"rate-test-model-1769a3-kserve-546f5d5c7f-cswsl\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") " pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.756217 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.756133    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:17.890578 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:17.890555    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"]
Aug 04 00:25:17.892891 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:25:17.892865    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b91d063_9324_4ce4_bb7c_c2790c702632.slice/crio-082c25b280660d19f1953ec8e2a65d4bf024df0c6b5ceea4a3586b0d4115fd41 WatchSource:0}: Error finding container 082c25b280660d19f1953ec8e2a65d4bf024df0c6b5ceea4a3586b0d4115fd41: Status 404 returned error can't find the container with id 082c25b280660d19f1953ec8e2a65d4bf024df0c6b5ceea4a3586b0d4115fd41
Aug 04 00:25:18.174969 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:18.174884    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl" event={"ID":"3b91d063-9324-4ce4-bb7c-c2790c702632","Type":"ContainerStarted","Data":"201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33"}
Aug 04 00:25:18.174969 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:18.174939    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl" event={"ID":"3b91d063-9324-4ce4-bb7c-c2790c702632","Type":"ContainerStarted","Data":"082c25b280660d19f1953ec8e2a65d4bf024df0c6b5ceea4a3586b0d4115fd41"}
Aug 04 00:25:18.191531 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:18.191488    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl" podStartSLOduration=1.191473348 podStartE2EDuration="1.191473348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:25:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:25:18.190209834 +0000 UTC m=+3325.891916499" watchObservedRunningTime="2026-08-04 00:25:18.191473348 +0000 UTC m=+3325.893180012"
Aug 04 00:25:27.756401 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:27.756361    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:27.756891 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:27.756416    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:27.769320 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:27.769296    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:25:28.237742 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:25:28.237694    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:27:08.297218 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:08.297133    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"]
Aug 04 00:27:23.317535 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.317487    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl" podUID="3b91d063-9324-4ce4-bb7c-c2790c702632" containerName="main" containerID="cri-o://201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33" gracePeriod=45
Aug 04 00:27:23.577841 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.577773    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:27:23.741820 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.741788    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-tmp-dir\") pod \"3b91d063-9324-4ce4-bb7c-c2790c702632\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") "
Aug 04 00:27:23.742105 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.741853    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8hpp\" (UniqueName: \"kubernetes.io/projected/3b91d063-9324-4ce4-bb7c-c2790c702632-kube-api-access-w8hpp\") pod \"3b91d063-9324-4ce4-bb7c-c2790c702632\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") "
Aug 04 00:27:23.742105 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.741900    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-model-cache\") pod \"3b91d063-9324-4ce4-bb7c-c2790c702632\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") "
Aug 04 00:27:23.742105 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.741922    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b91d063-9324-4ce4-bb7c-c2790c702632-tls-certs\") pod \"3b91d063-9324-4ce4-bb7c-c2790c702632\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") "
Aug 04 00:27:23.742105 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.742075    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-home\") pod \"3b91d063-9324-4ce4-bb7c-c2790c702632\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") "
Aug 04 00:27:23.742105 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.742105    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-dshm\") pod \"3b91d063-9324-4ce4-bb7c-c2790c702632\" (UID: \"3b91d063-9324-4ce4-bb7c-c2790c702632\") "
Aug 04 00:27:23.742362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.742107    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b91d063-9324-4ce4-bb7c-c2790c702632" (UID: "3b91d063-9324-4ce4-bb7c-c2790c702632"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 00:27:23.742362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.742214    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b91d063-9324-4ce4-bb7c-c2790c702632" (UID: "3b91d063-9324-4ce4-bb7c-c2790c702632"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 00:27:23.742362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.742321    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-home" (OuterVolumeSpecName: "home") pod "3b91d063-9324-4ce4-bb7c-c2790c702632" (UID: "3b91d063-9324-4ce4-bb7c-c2790c702632"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 00:27:23.742514 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.742494    2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-home\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:27:23.742565 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.742522    2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-tmp-dir\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:27:23.742565 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.742539    2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-model-cache\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:27:23.744302 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.744280    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b91d063-9324-4ce4-bb7c-c2790c702632-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b91d063-9324-4ce4-bb7c-c2790c702632" (UID: "3b91d063-9324-4ce4-bb7c-c2790c702632"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:27:23.744378 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.744357    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b91d063-9324-4ce4-bb7c-c2790c702632-kube-api-access-w8hpp" (OuterVolumeSpecName: "kube-api-access-w8hpp") pod "3b91d063-9324-4ce4-bb7c-c2790c702632" (UID: "3b91d063-9324-4ce4-bb7c-c2790c702632"). InnerVolumeSpecName "kube-api-access-w8hpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:27:23.744417 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.744356    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-dshm" (OuterVolumeSpecName: "dshm") pod "3b91d063-9324-4ce4-bb7c-c2790c702632" (UID: "3b91d063-9324-4ce4-bb7c-c2790c702632"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 00:27:23.801310 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.801279    2553 generic.go:356] "Generic (PLEG): container finished" podID="3b91d063-9324-4ce4-bb7c-c2790c702632" containerID="201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33" exitCode=0
Aug 04 00:27:23.801493 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.801315    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"
Aug 04 00:27:23.801493 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.801370    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl" event={"ID":"3b91d063-9324-4ce4-bb7c-c2790c702632","Type":"ContainerDied","Data":"201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33"}
Aug 04 00:27:23.801493 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.801412    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl" event={"ID":"3b91d063-9324-4ce4-bb7c-c2790c702632","Type":"ContainerDied","Data":"082c25b280660d19f1953ec8e2a65d4bf024df0c6b5ceea4a3586b0d4115fd41"}
Aug 04 00:27:23.801493 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.801434    2553 scope.go:117] "RemoveContainer" containerID="201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33"
Aug 04 00:27:23.810861 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.810844    2553 scope.go:117] "RemoveContainer" containerID="201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33"
Aug 04 00:27:23.811097 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:27:23.811080    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33\": container with ID starting with 201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33 not found: ID does not exist" containerID="201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33"
Aug 04 00:27:23.811149 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.811106    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33"} err="failed to get container status \"201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33\": rpc error: code = NotFound desc = could not find container \"201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33\": container with ID starting with 201c7605e56884b36e292fded147dad187a163c525f1edd2ddf458a1c38d0e33 not found: ID does not exist"
Aug 04 00:27:23.822511 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.822487    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"]
Aug 04 00:27:23.824506 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.824487    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1769a3/rate-test-model-1769a3-kserve-546f5d5c7f-cswsl"]
Aug 04 00:27:23.843602 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.843529    2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b91d063-9324-4ce4-bb7c-c2790c702632-tls-certs\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:27:23.843602 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.843557    2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b91d063-9324-4ce4-bb7c-c2790c702632-dshm\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:27:23.843602 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:23.843571    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8hpp\" (UniqueName: \"kubernetes.io/projected/3b91d063-9324-4ce4-bb7c-c2790c702632-kube-api-access-w8hpp\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:27:24.765357 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:27:24.765320    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b91d063-9324-4ce4-bb7c-c2790c702632" path="/var/lib/kubelet/pods/3b91d063-9324-4ce4-bb7c-c2790c702632/volumes"
Aug 04 00:28:32.390028 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:32.389998    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nsb5x_51e56f31-11d1-4d11-8d7f-3193495da2e0/payload-processing/0.log"
Aug 04 00:28:36.749099 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.749064    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"]
Aug 04 00:28:36.749584 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.749569    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b91d063-9324-4ce4-bb7c-c2790c702632" containerName="main"
Aug 04 00:28:36.749634 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.749586    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b91d063-9324-4ce4-bb7c-c2790c702632" containerName="main"
Aug 04 00:28:36.749673 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.749649    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b91d063-9324-4ce4-bb7c-c2790c702632" containerName="main"
Aug 04 00:28:36.752624 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.752608    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.756244 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.756222    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-8c194a\"/\"ipp-route-8c194a-kserve-self-signed-certs\""
Aug 04 00:28:36.756244 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.756232    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-8c194a\"/\"openshift-service-ca.crt\""
Aug 04 00:28:36.756436 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.756252    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-8c194a\"/\"kube-root-ca.crt\""
Aug 04 00:28:36.756436 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.756281    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-8c194a\"/\"default-dockercfg-tnjh2\""
Aug 04 00:28:36.766846 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.766826    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"]
Aug 04 00:28:36.810519 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.810497    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adbc6eac-70fd-491c-a1d5-fff4d77de900-tls-certs\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.810630 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.810528    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-tmp-dir\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.810630 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.810549    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-dshm\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.810630 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.810607    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-home\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.810630 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.810623    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmww9\" (UniqueName: \"kubernetes.io/projected/adbc6eac-70fd-491c-a1d5-fff4d77de900-kube-api-access-qmww9\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.810836 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.810663    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-model-cache\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.911546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.911524    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmww9\" (UniqueName: \"kubernetes.io/projected/adbc6eac-70fd-491c-a1d5-fff4d77de900-kube-api-access-qmww9\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.911653 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.911562    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-model-cache\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.911653 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.911644    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adbc6eac-70fd-491c-a1d5-fff4d77de900-tls-certs\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.911787 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.911768    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-tmp-dir\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.911849 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.911808    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-dshm\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.911910 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.911860    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-home\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.912014 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.911992    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-model-cache\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.912123 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.912104    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-tmp-dir\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.912188 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.912175    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-home\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.914042 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.914025    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-dshm\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.914201 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.914184    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adbc6eac-70fd-491c-a1d5-fff4d77de900-tls-certs\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:36.919272 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:36.919248    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmww9\" (UniqueName: \"kubernetes.io/projected/adbc6eac-70fd-491c-a1d5-fff4d77de900-kube-api-access-qmww9\") pod \"ipp-route-8c194a-kserve-7d78644b8c-9z4r5\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") " pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:37.065822 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:37.065767    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:37.204458 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:37.204437    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"]
Aug 04 00:28:37.206404 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:28:37.206372    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadbc6eac_70fd_491c_a1d5_fff4d77de900.slice/crio-9afa72a1aa24ae24128217c3886c4e7afdb0624e7000a37c0948fedbf7999456 WatchSource:0}: Error finding container 9afa72a1aa24ae24128217c3886c4e7afdb0624e7000a37c0948fedbf7999456: Status 404 returned error can't find the container with id 9afa72a1aa24ae24128217c3886c4e7afdb0624e7000a37c0948fedbf7999456
Aug 04 00:28:38.183256 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:38.183222    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5" event={"ID":"adbc6eac-70fd-491c-a1d5-fff4d77de900","Type":"ContainerStarted","Data":"b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d"}
Aug 04 00:28:38.183725 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:38.183266    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5" event={"ID":"adbc6eac-70fd-491c-a1d5-fff4d77de900","Type":"ContainerStarted","Data":"9afa72a1aa24ae24128217c3886c4e7afdb0624e7000a37c0948fedbf7999456"}
Aug 04 00:28:38.199258 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:38.199205    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5" podStartSLOduration=2.199190441 podStartE2EDuration="2.199190441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:28:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:28:38.198871077 +0000 UTC m=+3525.900577741" watchObservedRunningTime="2026-08-04 00:28:38.199190441 +0000 UTC m=+3525.900897105"
Aug 04 00:28:47.066964 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:47.066933    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:47.067360 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:47.066987    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:47.079201 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:47.079181    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:28:47.241135 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:28:47.241109    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:29:15.188404 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:15.188368    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nsb5x_51e56f31-11d1-4d11-8d7f-3193495da2e0/payload-processing/0.log"
Aug 04 00:29:20.924628 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.924594    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"]
Aug 04 00:29:20.928433 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.928415    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:20.931104 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.931070    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-a51f42-dockercfg-qt2jv\""
Aug 04 00:29:20.931503 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.931481    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-a51f42\""
Aug 04 00:29:20.938925 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.938902    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"]
Aug 04 00:29:20.958849 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.958830    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"]
Aug 04 00:29:20.962342 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.962326    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:20.972048 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.972020    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"]
Aug 04 00:29:20.996753 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.996732    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a1295a7-efe9-4629-a054-9bab75a01c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr\" (UID: \"7a1295a7-efe9-4629-a054-9bab75a01c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:20.996854 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.996816    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpgr5\" (UniqueName: \"kubernetes.io/projected/d6288063-7fbf-49c4-90d7-f73f746ba4b4-kube-api-access-wpgr5\") pod \"payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92\" (UID: \"d6288063-7fbf-49c4-90d7-f73f746ba4b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:20.996854 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.996835    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd64l\" (UniqueName: \"kubernetes.io/projected/7a1295a7-efe9-4629-a054-9bab75a01c8b-kube-api-access-pd64l\") pod \"payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr\" (UID: \"7a1295a7-efe9-4629-a054-9bab75a01c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:20.996976 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:20.996861    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6288063-7fbf-49c4-90d7-f73f746ba4b4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92\" (UID: \"d6288063-7fbf-49c4-90d7-f73f746ba4b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:21.097429 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.097400    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6288063-7fbf-49c4-90d7-f73f746ba4b4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92\" (UID: \"d6288063-7fbf-49c4-90d7-f73f746ba4b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:21.097589 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.097472    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a1295a7-efe9-4629-a054-9bab75a01c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr\" (UID: \"7a1295a7-efe9-4629-a054-9bab75a01c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:21.097589 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.097534    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpgr5\" (UniqueName: \"kubernetes.io/projected/d6288063-7fbf-49c4-90d7-f73f746ba4b4-kube-api-access-wpgr5\") pod \"payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92\" (UID: \"d6288063-7fbf-49c4-90d7-f73f746ba4b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:21.097589 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.097550    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd64l\" (UniqueName: \"kubernetes.io/projected/7a1295a7-efe9-4629-a054-9bab75a01c8b-kube-api-access-pd64l\") pod \"payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr\" (UID: \"7a1295a7-efe9-4629-a054-9bab75a01c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:21.098201 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.098179    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6288063-7fbf-49c4-90d7-f73f746ba4b4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92\" (UID: \"d6288063-7fbf-49c4-90d7-f73f746ba4b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:21.098282 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.098202    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a1295a7-efe9-4629-a054-9bab75a01c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr\" (UID: \"7a1295a7-efe9-4629-a054-9bab75a01c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:21.106012 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.105989    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd64l\" (UniqueName: \"kubernetes.io/projected/7a1295a7-efe9-4629-a054-9bab75a01c8b-kube-api-access-pd64l\") pod \"payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr\" (UID: \"7a1295a7-efe9-4629-a054-9bab75a01c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:21.106184 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.106164    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpgr5\" (UniqueName: \"kubernetes.io/projected/d6288063-7fbf-49c4-90d7-f73f746ba4b4-kube-api-access-wpgr5\") pod \"payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92\" (UID: \"d6288063-7fbf-49c4-90d7-f73f746ba4b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:21.240314 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.240294    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:21.273411 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.273384    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:21.387780 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.387747    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"]
Aug 04 00:29:21.389075 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:29:21.389045    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a1295a7_efe9_4629_a054_9bab75a01c8b.slice/crio-cafc92921d3fd3ec93ac2e323922c5149630967a8b829c65ba9300b881ed1819 WatchSource:0}: Error finding container cafc92921d3fd3ec93ac2e323922c5149630967a8b829c65ba9300b881ed1819: Status 404 returned error can't find the container with id cafc92921d3fd3ec93ac2e323922c5149630967a8b829c65ba9300b881ed1819
Aug 04 00:29:21.409398 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.409347    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr" event={"ID":"7a1295a7-efe9-4629-a054-9bab75a01c8b","Type":"ContainerStarted","Data":"cafc92921d3fd3ec93ac2e323922c5149630967a8b829c65ba9300b881ed1819"}
Aug 04 00:29:21.429966 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:21.429940    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"]
Aug 04 00:29:21.431143 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:29:21.431104    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6288063_7fbf_49c4_90d7_f73f746ba4b4.slice/crio-9c642b7427e773abea04d44b4428b34b77a7e3d84c906c42251892fa9fa48678 WatchSource:0}: Error finding container 9c642b7427e773abea04d44b4428b34b77a7e3d84c906c42251892fa9fa48678: Status 404 returned error can't find the container with id 9c642b7427e773abea04d44b4428b34b77a7e3d84c906c42251892fa9fa48678
Aug 04 00:29:22.415326 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:22.415292    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr" event={"ID":"7a1295a7-efe9-4629-a054-9bab75a01c8b","Type":"ContainerStarted","Data":"a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b"}
Aug 04 00:29:22.415809 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:22.415397    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:22.416879 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:22.416858    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92" event={"ID":"d6288063-7fbf-49c4-90d7-f73f746ba4b4","Type":"ContainerStarted","Data":"5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6"}
Aug 04 00:29:22.417009 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:22.416893    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92" event={"ID":"d6288063-7fbf-49c4-90d7-f73f746ba4b4","Type":"ContainerStarted","Data":"9c642b7427e773abea04d44b4428b34b77a7e3d84c906c42251892fa9fa48678"}
Aug 04 00:29:22.417009 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:22.416954    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:22.432990 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:22.432935    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr" podStartSLOduration=2.432919505 podStartE2EDuration="2.432919505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:29:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:29:22.429971831 +0000 UTC m=+3570.131678496" watchObservedRunningTime="2026-08-04 00:29:22.432919505 +0000 UTC m=+3570.134626171"
Aug 04 00:29:22.446313 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:22.446267    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92" podStartSLOduration=2.446250961 podStartE2EDuration="2.446250961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:29:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:29:22.444550977 +0000 UTC m=+3570.146257642" watchObservedRunningTime="2026-08-04 00:29:22.446250961 +0000 UTC m=+3570.147957622"
Aug 04 00:29:33.424324 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:33.424292    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:33.424686 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:33.424367    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:50.691016 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:50.690976    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"]
Aug 04 00:29:50.691499 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:50.691296    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92" podUID="d6288063-7fbf-49c4-90d7-f73f746ba4b4" containerName="payload-processing" containerID="cri-o://5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6" gracePeriod=30
Aug 04 00:29:50.696524 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:50.696458    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"]
Aug 04 00:29:50.696813 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:50.696790    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr" podUID="7a1295a7-efe9-4629-a054-9bab75a01c8b" containerName="payload-pre-processing" containerID="cri-o://a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b" gracePeriod=30
Aug 04 00:29:50.976162 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:50.976138    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:50.979135 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:50.979116    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:51.059928 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.059904    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpgr5\" (UniqueName: \"kubernetes.io/projected/d6288063-7fbf-49c4-90d7-f73f746ba4b4-kube-api-access-wpgr5\") pod \"d6288063-7fbf-49c4-90d7-f73f746ba4b4\" (UID: \"d6288063-7fbf-49c4-90d7-f73f746ba4b4\") "
Aug 04 00:29:51.060056 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.059947    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6288063-7fbf-49c4-90d7-f73f746ba4b4-plugins-config-volume\") pod \"d6288063-7fbf-49c4-90d7-f73f746ba4b4\" (UID: \"d6288063-7fbf-49c4-90d7-f73f746ba4b4\") "
Aug 04 00:29:51.060056 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.059972    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd64l\" (UniqueName: \"kubernetes.io/projected/7a1295a7-efe9-4629-a054-9bab75a01c8b-kube-api-access-pd64l\") pod \"7a1295a7-efe9-4629-a054-9bab75a01c8b\" (UID: \"7a1295a7-efe9-4629-a054-9bab75a01c8b\") "
Aug 04 00:29:51.060130 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.060084    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a1295a7-efe9-4629-a054-9bab75a01c8b-plugins-config-volume\") pod \"7a1295a7-efe9-4629-a054-9bab75a01c8b\" (UID: \"7a1295a7-efe9-4629-a054-9bab75a01c8b\") "
Aug 04 00:29:51.060352 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.060327    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6288063-7fbf-49c4-90d7-f73f746ba4b4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d6288063-7fbf-49c4-90d7-f73f746ba4b4" (UID: "d6288063-7fbf-49c4-90d7-f73f746ba4b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:29:51.060457 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.060436    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a1295a7-efe9-4629-a054-9bab75a01c8b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a1295a7-efe9-4629-a054-9bab75a01c8b" (UID: "7a1295a7-efe9-4629-a054-9bab75a01c8b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:29:51.062186 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.062167    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6288063-7fbf-49c4-90d7-f73f746ba4b4-kube-api-access-wpgr5" (OuterVolumeSpecName: "kube-api-access-wpgr5") pod "d6288063-7fbf-49c4-90d7-f73f746ba4b4" (UID: "d6288063-7fbf-49c4-90d7-f73f746ba4b4"). InnerVolumeSpecName "kube-api-access-wpgr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:29:51.062261 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.062233    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a1295a7-efe9-4629-a054-9bab75a01c8b-kube-api-access-pd64l" (OuterVolumeSpecName: "kube-api-access-pd64l") pod "7a1295a7-efe9-4629-a054-9bab75a01c8b" (UID: "7a1295a7-efe9-4629-a054-9bab75a01c8b"). InnerVolumeSpecName "kube-api-access-pd64l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:29:51.161464 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.161439    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpgr5\" (UniqueName: \"kubernetes.io/projected/d6288063-7fbf-49c4-90d7-f73f746ba4b4-kube-api-access-wpgr5\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:29:51.161464 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.161461    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6288063-7fbf-49c4-90d7-f73f746ba4b4-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:29:51.161601 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.161470    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd64l\" (UniqueName: \"kubernetes.io/projected/7a1295a7-efe9-4629-a054-9bab75a01c8b-kube-api-access-pd64l\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:29:51.161601 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.161480    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a1295a7-efe9-4629-a054-9bab75a01c8b-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:29:51.573860 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.573827    2553 generic.go:356] "Generic (PLEG): container finished" podID="7a1295a7-efe9-4629-a054-9bab75a01c8b" containerID="a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b" exitCode=0
Aug 04 00:29:51.574026 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.573888    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"
Aug 04 00:29:51.574026 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.573910    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr" event={"ID":"7a1295a7-efe9-4629-a054-9bab75a01c8b","Type":"ContainerDied","Data":"a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b"}
Aug 04 00:29:51.574026 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.574010    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr" event={"ID":"7a1295a7-efe9-4629-a054-9bab75a01c8b","Type":"ContainerDied","Data":"cafc92921d3fd3ec93ac2e323922c5149630967a8b829c65ba9300b881ed1819"}
Aug 04 00:29:51.574208 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.574044    2553 scope.go:117] "RemoveContainer" containerID="a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b"
Aug 04 00:29:51.575200 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.575134    2553 generic.go:356] "Generic (PLEG): container finished" podID="d6288063-7fbf-49c4-90d7-f73f746ba4b4" containerID="5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6" exitCode=0
Aug 04 00:29:51.575200 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.575195    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"
Aug 04 00:29:51.575315 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.575214    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92" event={"ID":"d6288063-7fbf-49c4-90d7-f73f746ba4b4","Type":"ContainerDied","Data":"5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6"}
Aug 04 00:29:51.575315 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.575255    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92" event={"ID":"d6288063-7fbf-49c4-90d7-f73f746ba4b4","Type":"ContainerDied","Data":"9c642b7427e773abea04d44b4428b34b77a7e3d84c906c42251892fa9fa48678"}
Aug 04 00:29:51.583126 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.583100    2553 scope.go:117] "RemoveContainer" containerID="a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b"
Aug 04 00:29:51.583408 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:29:51.583385    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b\": container with ID starting with a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b not found: ID does not exist" containerID="a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b"
Aug 04 00:29:51.583497 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.583418    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b"} err="failed to get container status \"a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b\": rpc error: code = NotFound desc = could not find container \"a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b\": container with ID starting with a6e898e246ca9962e11953a182b3301ae982a6953c4f7ddd6e3e81fba863484b not found: ID does not exist"
Aug 04 00:29:51.583497 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.583442    2553 scope.go:117] "RemoveContainer" containerID="5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6"
Aug 04 00:29:51.592058 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.592040    2553 scope.go:117] "RemoveContainer" containerID="5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6"
Aug 04 00:29:51.592320 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:29:51.592301    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6\": container with ID starting with 5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6 not found: ID does not exist" containerID="5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6"
Aug 04 00:29:51.592367 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.592327    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6"} err="failed to get container status \"5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6\": rpc error: code = NotFound desc = could not find container \"5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6\": container with ID starting with 5dad19c8ab8a8ec4bc30b0c6d0ff4990e8922448d484b051722b15cd4348fff6 not found: ID does not exist"
Aug 04 00:29:51.599994 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.599971    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"]
Aug 04 00:29:51.606500 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.606481    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a51f42-65fbd55b4-bd2vr"]
Aug 04 00:29:51.615803 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.615781    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"]
Aug 04 00:29:51.619857 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:51.619833    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a51f42-864d9954b-9xq92"]
Aug 04 00:29:52.765747 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:52.765698    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a1295a7-efe9-4629-a054-9bab75a01c8b" path="/var/lib/kubelet/pods/7a1295a7-efe9-4629-a054-9bab75a01c8b/volumes"
Aug 04 00:29:52.766082 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:29:52.766056    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6288063-7fbf-49c4-90d7-f73f746ba4b4" path="/var/lib/kubelet/pods/d6288063-7fbf-49c4-90d7-f73f746ba4b4/volumes"
Aug 04 00:30:21.972349 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:21.972316    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"]
Aug 04 00:30:36.991451 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:36.991398    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5" podUID="adbc6eac-70fd-491c-a1d5-fff4d77de900" containerName="main" containerID="cri-o://b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d" gracePeriod=45
Aug 04 00:30:37.243599 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.243546    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:30:37.386513 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.386488    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-dshm\") pod \"adbc6eac-70fd-491c-a1d5-fff4d77de900\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") "
Aug 04 00:30:37.386691 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.386526    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-tmp-dir\") pod \"adbc6eac-70fd-491c-a1d5-fff4d77de900\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") "
Aug 04 00:30:37.386691 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.386558    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmww9\" (UniqueName: \"kubernetes.io/projected/adbc6eac-70fd-491c-a1d5-fff4d77de900-kube-api-access-qmww9\") pod \"adbc6eac-70fd-491c-a1d5-fff4d77de900\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") "
Aug 04 00:30:37.386691 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.386591    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adbc6eac-70fd-491c-a1d5-fff4d77de900-tls-certs\") pod \"adbc6eac-70fd-491c-a1d5-fff4d77de900\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") "
Aug 04 00:30:37.386893 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.386797    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-home\") pod \"adbc6eac-70fd-491c-a1d5-fff4d77de900\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") "
Aug 04 00:30:37.386893 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.386855    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "adbc6eac-70fd-491c-a1d5-fff4d77de900" (UID: "adbc6eac-70fd-491c-a1d5-fff4d77de900"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 00:30:37.386893 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.386885    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-model-cache\") pod \"adbc6eac-70fd-491c-a1d5-fff4d77de900\" (UID: \"adbc6eac-70fd-491c-a1d5-fff4d77de900\") "
Aug 04 00:30:37.387081 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.387051    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-home" (OuterVolumeSpecName: "home") pod "adbc6eac-70fd-491c-a1d5-fff4d77de900" (UID: "adbc6eac-70fd-491c-a1d5-fff4d77de900"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 00:30:37.387160 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.387138    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-model-cache" (OuterVolumeSpecName: "model-cache") pod "adbc6eac-70fd-491c-a1d5-fff4d77de900" (UID: "adbc6eac-70fd-491c-a1d5-fff4d77de900"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 00:30:37.387460 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.387435    2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-tmp-dir\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:30:37.387551 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.387467    2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-home\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:30:37.387551 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.387481    2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-model-cache\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:30:37.388837 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.388809    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-dshm" (OuterVolumeSpecName: "dshm") pod "adbc6eac-70fd-491c-a1d5-fff4d77de900" (UID: "adbc6eac-70fd-491c-a1d5-fff4d77de900"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 00:30:37.388934 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.388831    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adbc6eac-70fd-491c-a1d5-fff4d77de900-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "adbc6eac-70fd-491c-a1d5-fff4d77de900" (UID: "adbc6eac-70fd-491c-a1d5-fff4d77de900"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:30:37.388934 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.388862    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adbc6eac-70fd-491c-a1d5-fff4d77de900-kube-api-access-qmww9" (OuterVolumeSpecName: "kube-api-access-qmww9") pod "adbc6eac-70fd-491c-a1d5-fff4d77de900" (UID: "adbc6eac-70fd-491c-a1d5-fff4d77de900"). InnerVolumeSpecName "kube-api-access-qmww9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:30:37.488481 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.488442    2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adbc6eac-70fd-491c-a1d5-fff4d77de900-dshm\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:30:37.488481 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.488479    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmww9\" (UniqueName: \"kubernetes.io/projected/adbc6eac-70fd-491c-a1d5-fff4d77de900-kube-api-access-qmww9\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:30:37.488673 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.488494    2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adbc6eac-70fd-491c-a1d5-fff4d77de900-tls-certs\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:30:37.824516 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.824478    2553 generic.go:356] "Generic (PLEG): container finished" podID="adbc6eac-70fd-491c-a1d5-fff4d77de900" containerID="b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d" exitCode=0
Aug 04 00:30:37.824686 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.824529    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"
Aug 04 00:30:37.824686 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.824569    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5" event={"ID":"adbc6eac-70fd-491c-a1d5-fff4d77de900","Type":"ContainerDied","Data":"b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d"}
Aug 04 00:30:37.824686 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.824622    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5" event={"ID":"adbc6eac-70fd-491c-a1d5-fff4d77de900","Type":"ContainerDied","Data":"9afa72a1aa24ae24128217c3886c4e7afdb0624e7000a37c0948fedbf7999456"}
Aug 04 00:30:37.824686 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.824644    2553 scope.go:117] "RemoveContainer" containerID="b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d"
Aug 04 00:30:37.833918 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.833900    2553 scope.go:117] "RemoveContainer" containerID="b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d"
Aug 04 00:30:37.834154 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:30:37.834135    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d\": container with ID starting with b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d not found: ID does not exist" containerID="b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d"
Aug 04 00:30:37.834211 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.834161    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d"} err="failed to get container status \"b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d\": rpc error: code = NotFound desc = could not find container \"b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d\": container with ID starting with b357c56b41feb5d88857cf6bcaf2ca299f7f068ad3f8e45c16a5f52f30fa977d not found: ID does not exist"
Aug 04 00:30:37.846104 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.845976    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"]
Aug 04 00:30:37.847931 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:37.847913    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5"]
Aug 04 00:30:38.764944 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:38.764912    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adbc6eac-70fd-491c-a1d5-fff4d77de900" path="/var/lib/kubelet/pods/adbc6eac-70fd-491c-a1d5-fff4d77de900/volumes"
Aug 04 00:30:42.231296 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:30:42.231247    2553 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-ipp-a-8c194a/ipp-route-8c194a-kserve-7d78644b8c-9z4r5" podUID="adbc6eac-70fd-491c-a1d5-fff4d77de900" containerName="main" probeResult="failure" output="Get \"https://10.133.0.78:8000/ready\": context deadline exceeded"
Aug 04 00:31:34.010250 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.010217    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"]
Aug 04 00:31:34.010845 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.010511    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb" podUID="1047395c-83fc-49f4-839f-3df342b461e2" containerName="maas-api" containerID="cri-o://f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6" gracePeriod=30
Aug 04 00:31:34.070329 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.070297    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"]
Aug 04 00:31:34.070563 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.070521    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz" podUID="95a55fb2-cc4f-4404-9a64-8fa13af10d06" containerName="payload-processing" containerID="cri-o://7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77" gracePeriod=30
Aug 04 00:31:34.077569 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.077538    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"]
Aug 04 00:31:34.077872 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.077825    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42" podUID="8f3a85b9-5936-430b-ac98-c0c7499232b5" containerName="payload-pre-processing" containerID="cri-o://b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8" gracePeriod=30
Aug 04 00:31:34.152351 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.152314    2553 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb" podUID="1047395c-83fc-49f4-839f-3df342b461e2" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.71:8443/health\": dial tcp 10.133.0.71:8443: connect: connection refused"
Aug 04 00:31:34.369767 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.369742    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:31:34.373561 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.373535    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:31:34.382461 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.382441    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:31:34.398867 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.398838    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1047395c-83fc-49f4-839f-3df342b461e2-openshift-service-ca\") pod \"1047395c-83fc-49f4-839f-3df342b461e2\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") "
Aug 04 00:31:34.399049 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.398885    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4fmh\" (UniqueName: \"kubernetes.io/projected/95a55fb2-cc4f-4404-9a64-8fa13af10d06-kube-api-access-b4fmh\") pod \"95a55fb2-cc4f-4404-9a64-8fa13af10d06\" (UID: \"95a55fb2-cc4f-4404-9a64-8fa13af10d06\") "
Aug 04 00:31:34.399049 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.398903    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mg7b\" (UniqueName: \"kubernetes.io/projected/1047395c-83fc-49f4-839f-3df342b461e2-kube-api-access-7mg7b\") pod \"1047395c-83fc-49f4-839f-3df342b461e2\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") "
Aug 04 00:31:34.399049 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.398968    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1047395c-83fc-49f4-839f-3df342b461e2-maas-api-tls\") pod \"1047395c-83fc-49f4-839f-3df342b461e2\" (UID: \"1047395c-83fc-49f4-839f-3df342b461e2\") "
Aug 04 00:31:34.399049 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.399031    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95a55fb2-cc4f-4404-9a64-8fa13af10d06-plugins-config-volume\") pod \"95a55fb2-cc4f-4404-9a64-8fa13af10d06\" (UID: \"95a55fb2-cc4f-4404-9a64-8fa13af10d06\") "
Aug 04 00:31:34.399281 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.399255    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1047395c-83fc-49f4-839f-3df342b461e2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1047395c-83fc-49f4-839f-3df342b461e2" (UID: "1047395c-83fc-49f4-839f-3df342b461e2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:31:34.399502 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.399481    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1047395c-83fc-49f4-839f-3df342b461e2-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:34.399697 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.399490    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95a55fb2-cc4f-4404-9a64-8fa13af10d06-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95a55fb2-cc4f-4404-9a64-8fa13af10d06" (UID: "95a55fb2-cc4f-4404-9a64-8fa13af10d06"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:31:34.402024 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.401984    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a55fb2-cc4f-4404-9a64-8fa13af10d06-kube-api-access-b4fmh" (OuterVolumeSpecName: "kube-api-access-b4fmh") pod "95a55fb2-cc4f-4404-9a64-8fa13af10d06" (UID: "95a55fb2-cc4f-4404-9a64-8fa13af10d06"). InnerVolumeSpecName "kube-api-access-b4fmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:31:34.402473 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.402442    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1047395c-83fc-49f4-839f-3df342b461e2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1047395c-83fc-49f4-839f-3df342b461e2" (UID: "1047395c-83fc-49f4-839f-3df342b461e2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:31:34.402789 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.402765    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1047395c-83fc-49f4-839f-3df342b461e2-kube-api-access-7mg7b" (OuterVolumeSpecName: "kube-api-access-7mg7b") pod "1047395c-83fc-49f4-839f-3df342b461e2" (UID: "1047395c-83fc-49f4-839f-3df342b461e2"). InnerVolumeSpecName "kube-api-access-7mg7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:31:34.500463 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.500431    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f3a85b9-5936-430b-ac98-c0c7499232b5-plugins-config-volume\") pod \"8f3a85b9-5936-430b-ac98-c0c7499232b5\" (UID: \"8f3a85b9-5936-430b-ac98-c0c7499232b5\") "
Aug 04 00:31:34.500653 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.500480    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m86h\" (UniqueName: \"kubernetes.io/projected/8f3a85b9-5936-430b-ac98-c0c7499232b5-kube-api-access-6m86h\") pod \"8f3a85b9-5936-430b-ac98-c0c7499232b5\" (UID: \"8f3a85b9-5936-430b-ac98-c0c7499232b5\") "
Aug 04 00:31:34.500930 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.500909    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1047395c-83fc-49f4-839f-3df342b461e2-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:34.500930 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.500919    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f3a85b9-5936-430b-ac98-c0c7499232b5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8f3a85b9-5936-430b-ac98-c0c7499232b5" (UID: "8f3a85b9-5936-430b-ac98-c0c7499232b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:31:34.501085 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.500937    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95a55fb2-cc4f-4404-9a64-8fa13af10d06-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:34.501085 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.500951    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4fmh\" (UniqueName: \"kubernetes.io/projected/95a55fb2-cc4f-4404-9a64-8fa13af10d06-kube-api-access-b4fmh\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:34.501085 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.500964    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mg7b\" (UniqueName: \"kubernetes.io/projected/1047395c-83fc-49f4-839f-3df342b461e2-kube-api-access-7mg7b\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:34.502787 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.502765    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f3a85b9-5936-430b-ac98-c0c7499232b5-kube-api-access-6m86h" (OuterVolumeSpecName: "kube-api-access-6m86h") pod "8f3a85b9-5936-430b-ac98-c0c7499232b5" (UID: "8f3a85b9-5936-430b-ac98-c0c7499232b5"). InnerVolumeSpecName "kube-api-access-6m86h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:31:34.601531 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.601449    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f3a85b9-5936-430b-ac98-c0c7499232b5-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:34.601531 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:34.601478    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m86h\" (UniqueName: \"kubernetes.io/projected/8f3a85b9-5936-430b-ac98-c0c7499232b5-kube-api-access-6m86h\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:35.114623 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.114588    2553 generic.go:356] "Generic (PLEG): container finished" podID="1047395c-83fc-49f4-839f-3df342b461e2" containerID="f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6" exitCode=0
Aug 04 00:31:35.115089 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.114634    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"
Aug 04 00:31:35.115089 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.114664    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb" event={"ID":"1047395c-83fc-49f4-839f-3df342b461e2","Type":"ContainerDied","Data":"f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6"}
Aug 04 00:31:35.115089 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.114734    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb" event={"ID":"1047395c-83fc-49f4-839f-3df342b461e2","Type":"ContainerDied","Data":"909adb6e8c49cb08a9977abc05f83c3ff66f1d944d5b46c24d6bf7146a0c9307"}
Aug 04 00:31:35.115089 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.114740    2553 scope.go:117] "RemoveContainer" containerID="f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6"
Aug 04 00:31:35.116193 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.116066    2553 generic.go:356] "Generic (PLEG): container finished" podID="95a55fb2-cc4f-4404-9a64-8fa13af10d06" containerID="7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77" exitCode=0
Aug 04 00:31:35.116193 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.116102    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz" event={"ID":"95a55fb2-cc4f-4404-9a64-8fa13af10d06","Type":"ContainerDied","Data":"7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77"}
Aug 04 00:31:35.116193 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.116123    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"
Aug 04 00:31:35.116193 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.116159    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz" event={"ID":"95a55fb2-cc4f-4404-9a64-8fa13af10d06","Type":"ContainerDied","Data":"f77302c3c2b62919be29707fd55b3e3a699007e97cb3a209f72bffbffd6a36fb"}
Aug 04 00:31:35.117776 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.117756    2553 generic.go:356] "Generic (PLEG): container finished" podID="8f3a85b9-5936-430b-ac98-c0c7499232b5" containerID="b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8" exitCode=0
Aug 04 00:31:35.117992 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.117782    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42" event={"ID":"8f3a85b9-5936-430b-ac98-c0c7499232b5","Type":"ContainerDied","Data":"b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8"}
Aug 04 00:31:35.117992 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.117798    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"
Aug 04 00:31:35.117992 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.117834    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42" event={"ID":"8f3a85b9-5936-430b-ac98-c0c7499232b5","Type":"ContainerDied","Data":"78c6ed71c06c49d8c73faa02de5d3239eb01db0f91ee5daad1450c653c55a675"}
Aug 04 00:31:35.124771 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.124744    2553 scope.go:117] "RemoveContainer" containerID="f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6"
Aug 04 00:31:35.125038 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:31:35.125020    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6\": container with ID starting with f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6 not found: ID does not exist" containerID="f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6"
Aug 04 00:31:35.125102 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.125048    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6"} err="failed to get container status \"f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6\": rpc error: code = NotFound desc = could not find container \"f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6\": container with ID starting with f2fa81bccad09580450bcba5c5012dc6ee1e60fa96f271555fdad5dc8889b3a6 not found: ID does not exist"
Aug 04 00:31:35.125102 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.125064    2553 scope.go:117] "RemoveContainer" containerID="7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77"
Aug 04 00:31:35.132825 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.132809    2553 scope.go:117] "RemoveContainer" containerID="7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77"
Aug 04 00:31:35.133079 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:31:35.133059    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77\": container with ID starting with 7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77 not found: ID does not exist" containerID="7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77"
Aug 04 00:31:35.133156 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.133089    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77"} err="failed to get container status \"7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77\": rpc error: code = NotFound desc = could not find container \"7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77\": container with ID starting with 7e2b5fc1d77c8a6f03bc4fcd9fb5806628f58edf67549d2593b865155fba0f77 not found: ID does not exist"
Aug 04 00:31:35.133156 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.133111    2553 scope.go:117] "RemoveContainer" containerID="b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8"
Aug 04 00:31:35.138524 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.138506    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"]
Aug 04 00:31:35.141464 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.141442    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1769a3-5445767f68-qdp42"]
Aug 04 00:31:35.142112 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.142088    2553 scope.go:117] "RemoveContainer" containerID="b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8"
Aug 04 00:31:35.142351 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:31:35.142334    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8\": container with ID starting with b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8 not found: ID does not exist" containerID="b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8"
Aug 04 00:31:35.142405 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.142358    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8"} err="failed to get container status \"b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8\": rpc error: code = NotFound desc = could not find container \"b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8\": container with ID starting with b94a36d0c12969adb15a63046ca70084b9d8e5574e11ce412ccc8a5211eb78b8 not found: ID does not exist"
Aug 04 00:31:35.156079 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.156059    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"]
Aug 04 00:31:35.160084 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.160062    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1769a3-56d687d5bb-466wz"]
Aug 04 00:31:35.170130 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.170107    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"]
Aug 04 00:31:35.173360 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:35.173342    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1769a3-79864db5c-n5ggb"]
Aug 04 00:31:36.764586 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:36.764549    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1047395c-83fc-49f4-839f-3df342b461e2" path="/var/lib/kubelet/pods/1047395c-83fc-49f4-839f-3df342b461e2/volumes"
Aug 04 00:31:36.765080 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:36.765063    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f3a85b9-5936-430b-ac98-c0c7499232b5" path="/var/lib/kubelet/pods/8f3a85b9-5936-430b-ac98-c0c7499232b5/volumes"
Aug 04 00:31:36.765499 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:36.765486    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95a55fb2-cc4f-4404-9a64-8fa13af10d06" path="/var/lib/kubelet/pods/95a55fb2-cc4f-4404-9a64-8fa13af10d06/volumes"
Aug 04 00:31:51.724148 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:51.724116    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"]
Aug 04 00:31:51.724774 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:51.724345    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm" podUID="cd833c55-3b2d-42d4-96b3-27d799bde070" containerName="maas-api" containerID="cri-o://35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e" gracePeriod=30
Aug 04 00:31:51.782087 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:51.782049    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"]
Aug 04 00:31:51.782347 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:51.782315    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6" podUID="78ce4994-95b0-4893-8d14-9f8b0c49b416" containerName="payload-processing" containerID="cri-o://5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f" gracePeriod=30
Aug 04 00:31:51.789805 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:51.789344    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"]
Aug 04 00:31:51.789805 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:51.789591    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2" podUID="3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5" containerName="payload-pre-processing" containerID="cri-o://dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b" gracePeriod=30
Aug 04 00:31:52.080387 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.080359    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:31:52.083348 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.083325    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:31:52.097351 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.097321    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:31:52.169443 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.169401    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd833c55-3b2d-42d4-96b3-27d799bde070-openshift-service-ca\") pod \"cd833c55-3b2d-42d4-96b3-27d799bde070\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") "
Aug 04 00:31:52.169656 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.169475    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fbnm\" (UniqueName: \"kubernetes.io/projected/78ce4994-95b0-4893-8d14-9f8b0c49b416-kube-api-access-6fbnm\") pod \"78ce4994-95b0-4893-8d14-9f8b0c49b416\" (UID: \"78ce4994-95b0-4893-8d14-9f8b0c49b416\") "
Aug 04 00:31:52.169725 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.169669    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrcs6\" (UniqueName: \"kubernetes.io/projected/cd833c55-3b2d-42d4-96b3-27d799bde070-kube-api-access-wrcs6\") pod \"cd833c55-3b2d-42d4-96b3-27d799bde070\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") "
Aug 04 00:31:52.169778 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.169765    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd833c55-3b2d-42d4-96b3-27d799bde070-maas-api-tls\") pod \"cd833c55-3b2d-42d4-96b3-27d799bde070\" (UID: \"cd833c55-3b2d-42d4-96b3-27d799bde070\") "
Aug 04 00:31:52.169829 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.169811    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78ce4994-95b0-4893-8d14-9f8b0c49b416-plugins-config-volume\") pod \"78ce4994-95b0-4893-8d14-9f8b0c49b416\" (UID: \"78ce4994-95b0-4893-8d14-9f8b0c49b416\") "
Aug 04 00:31:52.169882 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.169842    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd833c55-3b2d-42d4-96b3-27d799bde070-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cd833c55-3b2d-42d4-96b3-27d799bde070" (UID: "cd833c55-3b2d-42d4-96b3-27d799bde070"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:31:52.170175 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.170144    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd833c55-3b2d-42d4-96b3-27d799bde070-openshift-service-ca\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:52.170423 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.170220    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78ce4994-95b0-4893-8d14-9f8b0c49b416-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "78ce4994-95b0-4893-8d14-9f8b0c49b416" (UID: "78ce4994-95b0-4893-8d14-9f8b0c49b416"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:31:52.171810 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.171785    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd833c55-3b2d-42d4-96b3-27d799bde070-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cd833c55-3b2d-42d4-96b3-27d799bde070" (UID: "cd833c55-3b2d-42d4-96b3-27d799bde070"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 00:31:52.171892 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.171830    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78ce4994-95b0-4893-8d14-9f8b0c49b416-kube-api-access-6fbnm" (OuterVolumeSpecName: "kube-api-access-6fbnm") pod "78ce4994-95b0-4893-8d14-9f8b0c49b416" (UID: "78ce4994-95b0-4893-8d14-9f8b0c49b416"). InnerVolumeSpecName "kube-api-access-6fbnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:31:52.171892 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.171832    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd833c55-3b2d-42d4-96b3-27d799bde070-kube-api-access-wrcs6" (OuterVolumeSpecName: "kube-api-access-wrcs6") pod "cd833c55-3b2d-42d4-96b3-27d799bde070" (UID: "cd833c55-3b2d-42d4-96b3-27d799bde070"). InnerVolumeSpecName "kube-api-access-wrcs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:31:52.215043 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.215009    2553 generic.go:356] "Generic (PLEG): container finished" podID="78ce4994-95b0-4893-8d14-9f8b0c49b416" containerID="5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f" exitCode=0
Aug 04 00:31:52.215231 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.215066    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"
Aug 04 00:31:52.215231 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.215092    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6" event={"ID":"78ce4994-95b0-4893-8d14-9f8b0c49b416","Type":"ContainerDied","Data":"5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f"}
Aug 04 00:31:52.215231 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.215189    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6" event={"ID":"78ce4994-95b0-4893-8d14-9f8b0c49b416","Type":"ContainerDied","Data":"e109f91878b5e3b9cf2a3ed36686e911b26588d11918727d6ec4175749222015"}
Aug 04 00:31:52.215231 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.215222    2553 scope.go:117] "RemoveContainer" containerID="5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f"
Aug 04 00:31:52.216524 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.216504    2553 generic.go:356] "Generic (PLEG): container finished" podID="3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5" containerID="dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b" exitCode=0
Aug 04 00:31:52.216632 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.216545    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"
Aug 04 00:31:52.216632 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.216582    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2" event={"ID":"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5","Type":"ContainerDied","Data":"dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b"}
Aug 04 00:31:52.216761 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.216630    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2" event={"ID":"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5","Type":"ContainerDied","Data":"9cad16ffb4280de29f68cfec203e410c1ee890ce406f58802556b14af9cf7e1d"}
Aug 04 00:31:52.218185 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.218161    2553 generic.go:356] "Generic (PLEG): container finished" podID="cd833c55-3b2d-42d4-96b3-27d799bde070" containerID="35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e" exitCode=0
Aug 04 00:31:52.218304 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.218199    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"
Aug 04 00:31:52.218304 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.218241    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm" event={"ID":"cd833c55-3b2d-42d4-96b3-27d799bde070","Type":"ContainerDied","Data":"35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e"}
Aug 04 00:31:52.218304 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.218299    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm" event={"ID":"cd833c55-3b2d-42d4-96b3-27d799bde070","Type":"ContainerDied","Data":"310bf257c70882b6b1669525cf4c911ba7d49769c59cb1a671a78337d5817b5e"}
Aug 04 00:31:52.231266 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.231251    2553 scope.go:117] "RemoveContainer" containerID="5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f"
Aug 04 00:31:52.231542 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:31:52.231525    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f\": container with ID starting with 5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f not found: ID does not exist" containerID="5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f"
Aug 04 00:31:52.231588 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.231550    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f"} err="failed to get container status \"5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f\": rpc error: code = NotFound desc = could not find container \"5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f\": container with ID starting with 5956abbcac21896f165a4991e7caad24c2d0569ca536dfa9cc4a08abf28cdd5f not found: ID does not exist"
Aug 04 00:31:52.231588 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.231564    2553 scope.go:117] "RemoveContainer" containerID="dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b"
Aug 04 00:31:52.239549 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.239528    2553 scope.go:117] "RemoveContainer" containerID="dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b"
Aug 04 00:31:52.239832 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:31:52.239813    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b\": container with ID starting with dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b not found: ID does not exist" containerID="dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b"
Aug 04 00:31:52.239900 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.239839    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b"} err="failed to get container status \"dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b\": rpc error: code = NotFound desc = could not find container \"dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b\": container with ID starting with dbf8319e230357bd5d17c4952bdfeb98c81b09190b19c187390fa84f29c1352b not found: ID does not exist"
Aug 04 00:31:52.239900 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.239856    2553 scope.go:117] "RemoveContainer" containerID="35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e"
Aug 04 00:31:52.245830 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.245760    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"]
Aug 04 00:31:52.248059 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.248043    2553 scope.go:117] "RemoveContainer" containerID="35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e"
Aug 04 00:31:52.248367 ip-10-0-143-183 kubenswrapper[2553]: E0804 00:31:52.248348    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e\": container with ID starting with 35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e not found: ID does not exist" containerID="35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e"
Aug 04 00:31:52.248422 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.248374    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e"} err="failed to get container status \"35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e\": rpc error: code = NotFound desc = could not find container \"35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e\": container with ID starting with 35f45a125cc1d6fc2162f7c103467abc0d7eee72a69028d8a985fdc79659171e not found: ID does not exist"
Aug 04 00:31:52.250981 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.250962    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-98f978-76657fc476-m8vgm"]
Aug 04 00:31:52.261250 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.261227    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"]
Aug 04 00:31:52.265281 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.265262    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-98f978-6d8d666b4b-k4wh6"]
Aug 04 00:31:52.270440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.270422    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-plugins-config-volume\") pod \"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5\" (UID: \"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5\") "
Aug 04 00:31:52.270546 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.270528    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbxch\" (UniqueName: \"kubernetes.io/projected/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-kube-api-access-rbxch\") pod \"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5\" (UID: \"3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5\") "
Aug 04 00:31:52.270760 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.270741    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5" (UID: "3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 00:31:52.270899 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.270883    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrcs6\" (UniqueName: \"kubernetes.io/projected/cd833c55-3b2d-42d4-96b3-27d799bde070-kube-api-access-wrcs6\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:52.270964 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.270906    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd833c55-3b2d-42d4-96b3-27d799bde070-maas-api-tls\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:52.270964 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.270921    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78ce4994-95b0-4893-8d14-9f8b0c49b416-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:52.270964 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.270935    2553 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-plugins-config-volume\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:52.270964 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.270948    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fbnm\" (UniqueName: \"kubernetes.io/projected/78ce4994-95b0-4893-8d14-9f8b0c49b416-kube-api-access-6fbnm\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:52.272627 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.272607    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-kube-api-access-rbxch" (OuterVolumeSpecName: "kube-api-access-rbxch") pod "3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5" (UID: "3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5"). InnerVolumeSpecName "kube-api-access-rbxch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 00:31:52.372133 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.372096    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbxch\" (UniqueName: \"kubernetes.io/projected/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5-kube-api-access-rbxch\") on node \"ip-10-0-143-183.ec2.internal\" DevicePath \"\""
Aug 04 00:31:52.537567 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.537538    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"]
Aug 04 00:31:52.541534 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.541506    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-98f978-857744bfbd-wlfd2"]
Aug 04 00:31:52.766434 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.766405    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5" path="/var/lib/kubelet/pods/3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5/volumes"
Aug 04 00:31:52.766793 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.766780    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78ce4994-95b0-4893-8d14-9f8b0c49b416" path="/var/lib/kubelet/pods/78ce4994-95b0-4893-8d14-9f8b0c49b416/volumes"
Aug 04 00:31:52.767137 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:31:52.767124    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd833c55-3b2d-42d4-96b3-27d799bde070" path="/var/lib/kubelet/pods/cd833c55-3b2d-42d4-96b3-27d799bde070/volumes"
Aug 04 00:32:04.596426 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:04.596394    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-96bff6bb9-hz6tn_570ee62e-722b-4af2-a1a8-f4093fe0e5bf/manager/0.log"
Aug 04 00:32:05.071647 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:05.071621    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6869c6d9bb-b2lh2_29498929-93a1-4db4-9b8e-d3d0c3deb906/copy-manifests/0.log"
Aug 04 00:32:05.078039 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:05.078008    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6869c6d9bb-b2lh2_29498929-93a1-4db4-9b8e-d3d0c3deb906/manager/0.log"
Aug 04 00:32:05.506021 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:05.505989    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-69bc574f56-4s4ld_38fab0b3-ea17-4bef-9d0c-a8515fc266e4/maas-api/0.log"
Aug 04 00:32:07.012682 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:07.012652    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-pwks2_fec9d7c3-08e6-4473-a842-e3acee90dac6/kuadrant-console-plugin/0.log"
Aug 04 00:32:08.038183 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:08.038154    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd58c6859-d5n7t_4a7db60a-8023-4e79-a1c4-9066642292d4/kube-auth-proxy/0.log"
Aug 04 00:32:08.263644 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:08.263613    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-95kbd_2864afa1-5450-409d-b876-f85b6ed727a0/payload-pre-processing/0.log"
Aug 04 00:32:08.376216 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:08.376136    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nsb5x_51e56f31-11d1-4d11-8d7f-3193495da2e0/payload-processing/0.log"
Aug 04 00:32:08.488504 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:08.488468    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f79c65b86-kvn79_3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e/router/0.log"
Aug 04 00:32:08.934848 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:08.934821    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-29lnb_fb4c8781-6f5a-4d6f-87bd-c030fd1c80d4/main/0.log"
Aug 04 00:32:09.044995 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:09.044957    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hkqlp_2f818d1b-10d2-40a5-9d66-4fe74d7b133f/main/0.log"
Aug 04 00:32:09.269199 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:09.269114    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-pknlc_6bacf1a4-32eb-4388-9ffa-25d0469275dd/main/0.log"
Aug 04 00:32:18.087807 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:18.087774    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w526z_57500f92-3fda-430a-bbc0-37e3d091d27f/global-pull-secret-syncer/0.log"
Aug 04 00:32:18.200697 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:18.200663    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cw2jt_6a69b3b6-18d1-4f41-8529-ad97331b0aa6/konnectivity-agent/0.log"
Aug 04 00:32:18.269308 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:18.269273    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-183.ec2.internal_43e10beb2c51db4c48da59eafdb1be9c/haproxy/0.log"
Aug 04 00:32:23.458012 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:23.457972    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-pwks2_fec9d7c3-08e6-4473-a842-e3acee90dac6/kuadrant-console-plugin/0.log"
Aug 04 00:32:25.139347 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.139317    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e492fc4-69ce-4cce-b5cd-74227c9cbdea/alertmanager/0.log"
Aug 04 00:32:25.159281 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.159254    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e492fc4-69ce-4cce-b5cd-74227c9cbdea/config-reloader/0.log"
Aug 04 00:32:25.181285 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.181263    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e492fc4-69ce-4cce-b5cd-74227c9cbdea/kube-rbac-proxy-web/0.log"
Aug 04 00:32:25.202744 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.202724    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e492fc4-69ce-4cce-b5cd-74227c9cbdea/kube-rbac-proxy/0.log"
Aug 04 00:32:25.229344 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.229325    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e492fc4-69ce-4cce-b5cd-74227c9cbdea/kube-rbac-proxy-metric/0.log"
Aug 04 00:32:25.257275 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.257216    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e492fc4-69ce-4cce-b5cd-74227c9cbdea/prom-label-proxy/0.log"
Aug 04 00:32:25.287873 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.287856    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e492fc4-69ce-4cce-b5cd-74227c9cbdea/init-config-reloader/0.log"
Aug 04 00:32:25.334514 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.334491    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-csljn_a01ef950-7a61-40ba-a9c0-1d95c8b7b573/cluster-monitoring-operator/0.log"
Aug 04 00:32:25.709786 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.709761    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wmvr5_cadbad4c-dec8-4dfd-959c-3570a1dd486f/node-exporter/0.log"
Aug 04 00:32:25.743896 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.743874    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wmvr5_cadbad4c-dec8-4dfd-959c-3570a1dd486f/kube-rbac-proxy/0.log"
Aug 04 00:32:25.762421 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.762401    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wmvr5_cadbad4c-dec8-4dfd-959c-3570a1dd486f/init-textfile/0.log"
Aug 04 00:32:25.870171 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.870144    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_73ea3c88-2594-443b-a799-1ef0c82103c1/prometheus/0.log"
Aug 04 00:32:25.891654 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.891397    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_73ea3c88-2594-443b-a799-1ef0c82103c1/config-reloader/0.log"
Aug 04 00:32:25.911362 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.911300    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_73ea3c88-2594-443b-a799-1ef0c82103c1/thanos-sidecar/0.log"
Aug 04 00:32:25.934220 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.934195    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_73ea3c88-2594-443b-a799-1ef0c82103c1/kube-rbac-proxy-web/0.log"
Aug 04 00:32:25.954388 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.954367    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_73ea3c88-2594-443b-a799-1ef0c82103c1/kube-rbac-proxy/0.log"
Aug 04 00:32:25.977158 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.977133    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_73ea3c88-2594-443b-a799-1ef0c82103c1/kube-rbac-proxy-thanos/0.log"
Aug 04 00:32:25.997230 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:25.997208    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_73ea3c88-2594-443b-a799-1ef0c82103c1/init-config-reloader/0.log"
Aug 04 00:32:26.051938 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:26.051915    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-4rfrq_9b64caa8-b147-47db-aba4-470d30ef0f48/prometheus-operator/0.log"
Aug 04 00:32:26.070505 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:26.070485    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-4rfrq_9b64caa8-b147-47db-aba4-470d30ef0f48/kube-rbac-proxy/0.log"
Aug 04 00:32:26.103286 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:26.103260    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-hqzdq_71ca6f7e-4eef-4260-8e66-1bfc9fd0de07/prometheus-operator-admission-webhook/0.log"
Aug 04 00:32:26.136130 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:26.136111    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9667fbd4d-f4xx4_8c6211f2-278b-4192-a4e9-1ca7f0a30797/telemeter-client/0.log"
Aug 04 00:32:26.159417 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:26.159386    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9667fbd4d-f4xx4_8c6211f2-278b-4192-a4e9-1ca7f0a30797/reload/0.log"
Aug 04 00:32:26.176464 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:26.176396    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9667fbd4d-f4xx4_8c6211f2-278b-4192-a4e9-1ca7f0a30797/kube-rbac-proxy/0.log"
Aug 04 00:32:28.353671 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.353638    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"]
Aug 04 00:32:28.354065 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354052    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78ce4994-95b0-4893-8d14-9f8b0c49b416" containerName="payload-processing"
Aug 04 00:32:28.354065 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354066    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="78ce4994-95b0-4893-8d14-9f8b0c49b416" containerName="payload-processing"
Aug 04 00:32:28.354140 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354077    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6288063-7fbf-49c4-90d7-f73f746ba4b4" containerName="payload-processing"
Aug 04 00:32:28.354140 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354082    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6288063-7fbf-49c4-90d7-f73f746ba4b4" containerName="payload-processing"
Aug 04 00:32:28.354140 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354091    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f3a85b9-5936-430b-ac98-c0c7499232b5" containerName="payload-pre-processing"
Aug 04 00:32:28.354140 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354097    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f3a85b9-5936-430b-ac98-c0c7499232b5" containerName="payload-pre-processing"
Aug 04 00:32:28.354140 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354103    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1047395c-83fc-49f4-839f-3df342b461e2" containerName="maas-api"
Aug 04 00:32:28.354140 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354108    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1047395c-83fc-49f4-839f-3df342b461e2" containerName="maas-api"
Aug 04 00:32:28.354140 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354117    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a1295a7-efe9-4629-a054-9bab75a01c8b" containerName="payload-pre-processing"
Aug 04 00:32:28.354140 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354123    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a1295a7-efe9-4629-a054-9bab75a01c8b" containerName="payload-pre-processing"
Aug 04 00:32:28.354140 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354133    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95a55fb2-cc4f-4404-9a64-8fa13af10d06" containerName="payload-processing"
Aug 04 00:32:28.354140 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354138    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a55fb2-cc4f-4404-9a64-8fa13af10d06" containerName="payload-processing"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354150    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd833c55-3b2d-42d4-96b3-27d799bde070" containerName="maas-api"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354156    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd833c55-3b2d-42d4-96b3-27d799bde070" containerName="maas-api"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354162    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adbc6eac-70fd-491c-a1d5-fff4d77de900" containerName="main"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354167    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="adbc6eac-70fd-491c-a1d5-fff4d77de900" containerName="main"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354175    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5" containerName="payload-pre-processing"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354180    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5" containerName="payload-pre-processing"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354238    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd833c55-3b2d-42d4-96b3-27d799bde070" containerName="maas-api"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354247    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f3a85b9-5936-430b-ac98-c0c7499232b5" containerName="payload-pre-processing"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354253    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a1295a7-efe9-4629-a054-9bab75a01c8b" containerName="payload-pre-processing"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354259    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="adbc6eac-70fd-491c-a1d5-fff4d77de900" containerName="main"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354266    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aa7e0b6-cb66-4f78-b9fc-2df4dee472d5" containerName="payload-pre-processing"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354272    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="78ce4994-95b0-4893-8d14-9f8b0c49b416" containerName="payload-processing"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354279    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6288063-7fbf-49c4-90d7-f73f746ba4b4" containerName="payload-processing"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354284    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="95a55fb2-cc4f-4404-9a64-8fa13af10d06" containerName="payload-processing"
Aug 04 00:32:28.354440 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.354290    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1047395c-83fc-49f4-839f-3df342b461e2" containerName="maas-api"
Aug 04 00:32:28.357512 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.357488    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.360144 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.360126    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b95cg\"/\"default-dockercfg-zqc6c\""
Aug 04 00:32:28.360276 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.360131    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b95cg\"/\"openshift-service-ca.crt\""
Aug 04 00:32:28.361189 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.361174    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b95cg\"/\"kube-root-ca.crt\""
Aug 04 00:32:28.367443 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.367418    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"]
Aug 04 00:32:28.516158 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.516129    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-podres\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.516311 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.516161    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shhl2\" (UniqueName: \"kubernetes.io/projected/76bfc0d3-4c97-4729-a658-57cdab5bb302-kube-api-access-shhl2\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.516311 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.516221    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-sys\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.516311 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.516294    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-lib-modules\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.516445 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.516322    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-proc\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.617005 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.616909    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-podres\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.617005 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.616947    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shhl2\" (UniqueName: \"kubernetes.io/projected/76bfc0d3-4c97-4729-a658-57cdab5bb302-kube-api-access-shhl2\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.617186 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.617083    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-sys\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.617186 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.617119    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-sys\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.617186 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.617089    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-podres\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.617186 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.617178    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-lib-modules\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.617327 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.617230    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-proc\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.617327 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.617276    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-lib-modules\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.617397 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.617326    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76bfc0d3-4c97-4729-a658-57cdab5bb302-proc\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.625048 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.625020    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shhl2\" (UniqueName: \"kubernetes.io/projected/76bfc0d3-4c97-4729-a658-57cdab5bb302-kube-api-access-shhl2\") pod \"perf-node-gather-daemonset-dpzjs\" (UID: \"76bfc0d3-4c97-4729-a658-57cdab5bb302\") " pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.668183 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.668155    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:28.802716 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.802680    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"]
Aug 04 00:32:28.805025 ip-10-0-143-183 kubenswrapper[2553]: W0804 00:32:28.804999    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod76bfc0d3_4c97_4729_a658_57cdab5bb302.slice/crio-8bd692fef90a27cfaf9a3f30106ee46bf86e5e5d3c5115a1d3605b53856fe31e WatchSource:0}: Error finding container 8bd692fef90a27cfaf9a3f30106ee46bf86e5e5d3c5115a1d3605b53856fe31e: Status 404 returned error can't find the container with id 8bd692fef90a27cfaf9a3f30106ee46bf86e5e5d3c5115a1d3605b53856fe31e
Aug 04 00:32:28.867429 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:28.867374    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-dq5z2_9a63ea75-d9ca-4604-ad9d-a3e2fe4368d6/download-server/0.log"
Aug 04 00:32:29.400396 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:29.400349    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs" event={"ID":"76bfc0d3-4c97-4729-a658-57cdab5bb302","Type":"ContainerStarted","Data":"60b73f7a2a7dc18e93a2083a77ae71751554250e159c2053eadc752e41b7110a"}
Aug 04 00:32:29.400913 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:29.400433    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs" event={"ID":"76bfc0d3-4c97-4729-a658-57cdab5bb302","Type":"ContainerStarted","Data":"8bd692fef90a27cfaf9a3f30106ee46bf86e5e5d3c5115a1d3605b53856fe31e"}
Aug 04 00:32:29.400913 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:29.400491    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:29.417586 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:29.417543    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs" podStartSLOduration=1.417529266 podStartE2EDuration="1.417529266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:32:29.415408723 +0000 UTC m=+3757.117115389" watchObservedRunningTime="2026-08-04 00:32:29.417529266 +0000 UTC m=+3757.119235930"
Aug 04 00:32:30.593061 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:30.593026    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6mrf2_731c0106-b5c5-4a64-957d-38254cee1862/dns/0.log"
Aug 04 00:32:30.613646 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:30.613613    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6mrf2_731c0106-b5c5-4a64-957d-38254cee1862/kube-rbac-proxy/0.log"
Aug 04 00:32:30.785022 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:30.784996    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qm6s6_ced51f35-5c98-4126-a3b4-533675741b86/dns-node-resolver/0.log"
Aug 04 00:32:31.310007 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:31.309978    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29763360-5x87r_6414e19d-e976-458a-ade8-85a8f9bb2443/image-pruner/0.log"
Aug 04 00:32:31.426479 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:31.426442    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkhxt_79fe88bf-1fbb-4616-9dbb-3a3ccab24265/node-ca/0.log"
Aug 04 00:32:32.614691 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:32.614661    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd58c6859-d5n7t_4a7db60a-8023-4e79-a1c4-9066642292d4/kube-auth-proxy/0.log"
Aug 04 00:32:32.683542 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:32.683514    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-95kbd_2864afa1-5450-409d-b876-f85b6ed727a0/payload-pre-processing/0.log"
Aug 04 00:32:32.710449 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:32.710413    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nsb5x_51e56f31-11d1-4d11-8d7f-3193495da2e0/payload-processing/0.log"
Aug 04 00:32:32.731485 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:32.731462    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f79c65b86-kvn79_3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e/router/0.log"
Aug 04 00:32:33.550425 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:33.550397    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-np8h4_966fe46a-d62f-4d39-ae52-5e9e12b681c6/serve-healthcheck-canary/0.log"
Aug 04 00:32:34.049235 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:34.049206    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-zzb4f_2901f2fd-f8ce-47eb-a4c1-d0dba309961a/insights-operator/0.log"
Aug 04 00:32:34.049733 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:34.049716    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-zzb4f_2901f2fd-f8ce-47eb-a4c1-d0dba309961a/insights-operator/1.log"
Aug 04 00:32:34.209128 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:34.209097    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tttx8_8573ff4f-bab3-4d29-b4d1-d8d8a0941687/kube-rbac-proxy/0.log"
Aug 04 00:32:34.228340 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:34.228318    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tttx8_8573ff4f-bab3-4d29-b4d1-d8d8a0941687/exporter/0.log"
Aug 04 00:32:34.247811 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:34.247791    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tttx8_8573ff4f-bab3-4d29-b4d1-d8d8a0941687/extractor/0.log"
Aug 04 00:32:35.415378 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:35.415349    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b95cg/perf-node-gather-daemonset-dpzjs"
Aug 04 00:32:36.751890 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:36.751854    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-96bff6bb9-hz6tn_570ee62e-722b-4af2-a1a8-f4093fe0e5bf/manager/0.log"
Aug 04 00:32:37.043494 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:37.043412    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6869c6d9bb-b2lh2_29498929-93a1-4db4-9b8e-d3d0c3deb906/manager/0.log"
Aug 04 00:32:37.063388 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:37.063362    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6869c6d9bb-b2lh2_29498929-93a1-4db4-9b8e-d3d0c3deb906/copy-manifests/0.log"
Aug 04 00:32:38.564108 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:38.564080    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d64bf5b4d-6j4gt_1d347fef-80d3-4772-b33f-5202aa41f86a/manager/0.log"
Aug 04 00:32:39.438215 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:39.438184    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd58c6859-d5n7t_4a7db60a-8023-4e79-a1c4-9066642292d4/kube-auth-proxy/0.log"
Aug 04 00:32:39.495421 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:39.495340    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-95kbd_2864afa1-5450-409d-b876-f85b6ed727a0/payload-pre-processing/0.log"
Aug 04 00:32:39.515054 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:39.515030    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nsb5x_51e56f31-11d1-4d11-8d7f-3193495da2e0/payload-processing/0.log"
Aug 04 00:32:39.531636 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:39.531611    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f79c65b86-kvn79_3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e/router/0.log"
Aug 04 00:32:41.128647 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:41.128593    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd58c6859-d5n7t_4a7db60a-8023-4e79-a1c4-9066642292d4/kube-auth-proxy/0.log"
Aug 04 00:32:41.181355 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:41.181329    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-95kbd_2864afa1-5450-409d-b876-f85b6ed727a0/payload-pre-processing/0.log"
Aug 04 00:32:41.202398 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:41.202371    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nsb5x_51e56f31-11d1-4d11-8d7f-3193495da2e0/payload-processing/0.log"
Aug 04 00:32:41.217849 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:41.217823    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f79c65b86-kvn79_3a5cf9aa-d5bb-4aa2-9c9a-6c3ac2f3c93e/router/0.log"
Aug 04 00:32:43.873379 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:43.873295    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-pwwls_7cc92f64-d9da-42ae-8883-c1e5dc28ceea/kube-storage-version-migrator-operator/1.log"
Aug 04 00:32:43.874386 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:43.874368    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-pwwls_7cc92f64-d9da-42ae-8883-c1e5dc28ceea/kube-storage-version-migrator-operator/0.log"
Aug 04 00:32:45.176229 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:45.176198    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xcf2p_39619f91-b9cc-4d59-bcac-b0da494614ee/kube-multus-additional-cni-plugins/0.log"
Aug 04 00:32:45.195513 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:45.195486    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xcf2p_39619f91-b9cc-4d59-bcac-b0da494614ee/egress-router-binary-copy/0.log"
Aug 04 00:32:45.214656 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:45.214636    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xcf2p_39619f91-b9cc-4d59-bcac-b0da494614ee/cni-plugins/0.log"
Aug 04 00:32:45.234876 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:45.234858    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xcf2p_39619f91-b9cc-4d59-bcac-b0da494614ee/bond-cni-plugin/0.log"
Aug 04 00:32:45.254126 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:45.254103    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xcf2p_39619f91-b9cc-4d59-bcac-b0da494614ee/routeoverride-cni/0.log"
Aug 04 00:32:45.273384 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:45.273366    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xcf2p_39619f91-b9cc-4d59-bcac-b0da494614ee/whereabouts-cni-bincopy/0.log"
Aug 04 00:32:45.292100 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:45.292083    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xcf2p_39619f91-b9cc-4d59-bcac-b0da494614ee/whereabouts-cni/0.log"
Aug 04 00:32:45.359828 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:45.359805    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-g24wc_418e085f-c3bf-46eb-9ce1-9751c038c924/kube-multus/0.log"
Aug 04 00:32:45.465008 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:45.464984    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kld8m_5e420d03-b0de-4f89-936a-cdafff2847a9/network-metrics-daemon/0.log"
Aug 04 00:32:45.481933 ip-10-0-143-183 kubenswrapper[2553]: I0804 00:32:45.481911    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kld8m_5e420d03-b0de-4f89-936a-cdafff2847a9/kube-rbac-proxy/0.log"