Jul 23 15:48:55.259784 ip-10-0-140-230 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 15:48:55.564424 ip-10-0-140-230 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 15:48:55.564424 ip-10-0-140-230 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 15:48:55.564424 ip-10-0-140-230 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 15:48:55.564424 ip-10-0-140-230 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 15:48:55.564424 ip-10-0-140-230 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 15:48:55.565873 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.565725    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 15:48:55.570439 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570406    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 15:48:55.570647 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570514    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 15:48:55.570681 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570652    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 15:48:55.570681 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570658    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 15:48:55.570681 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570662    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 15:48:55.570681 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570665    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 15:48:55.570681 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570669    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 15:48:55.570681 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570671    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 15:48:55.570681 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570675    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 15:48:55.570681 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570678    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 15:48:55.570681 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570681    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 15:48:55.570681 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570685    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570688    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570691    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570694    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570697    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570700    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570703    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570705    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570708    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570711    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570714    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570716    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570719    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570721    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570724    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570727    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570729    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570732    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570735    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570737    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 15:48:55.570921 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570740    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570742    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570745    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570748    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570751    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570754    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570756    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570759    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570762    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570764    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570767    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570769    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570772    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570774    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570778    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570781    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570783    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570786    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570789    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570792    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 15:48:55.571389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570795    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570798    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570800    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570805    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570810    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570813    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570816    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570819    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570821    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570824    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570827    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570830    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570836    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570839    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570842    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570846    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570848    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570851    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570854    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 15:48:55.571926 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570857    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570860    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570862    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570866    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570870    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570873    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570875    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570880    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570883    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570885    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570888    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570891    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570893    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570897    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570900    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570903    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570905    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570925    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570930    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570934    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 15:48:55.572385 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570937    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570940    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570943    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570946    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570948    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570951    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570954    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570957    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.570959    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571723    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571731    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571735    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571737    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571740    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571743    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571746    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571749    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571752    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571755    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571759    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 15:48:55.572878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571761    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571764    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571766    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571770    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571772    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571775    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571780    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571783    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571785    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571788    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571791    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571794    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571796    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571799    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571801    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571804    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571807    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571810    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571812    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571815    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 15:48:55.573362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571817    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571820    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571823    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571826    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571828    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571831    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571833    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571836    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571838    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571841    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571844    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571848    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571852    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571855    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571857    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571861    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571863    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571866    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571869    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571872    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 15:48:55.573888 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571874    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571876    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571879    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571881    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571884    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571887    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571890    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571892    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571895    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571897    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571900    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571902    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571905    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571908    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571910    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571913    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571915    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571919    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571923    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571926    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 15:48:55.574383 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571929    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571932    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571935    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571938    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571941    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571944    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571946    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571949    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571951    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571954    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571956    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571959    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571962    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571964    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571967    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571969    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571973    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571977    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571980    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571983    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 15:48:55.574894 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571987    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571990    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571992    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571995    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.571998    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572000    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572003    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572006    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572479    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572486    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572490    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572494    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572497    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572500    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572504    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572507    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572510    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572512    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572515    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 15:48:55.575402 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572518    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572521    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572523    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572526    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572529    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572532    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572535    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572537    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572540    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572542    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572544    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572547    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572550    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572552    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572555    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572558    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572560    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572563    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572565    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 15:48:55.575905 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572568    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572570    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572573    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572576    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572578    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572581    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572583    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572585    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572588    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572591    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572594    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572597    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572599    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572602    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572604    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572607    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572612    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572614    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572617    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572620    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572623    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 15:48:55.576368 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572625    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572628    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572630    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572633    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572635    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572638    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572641    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572643    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572646    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572649    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572651    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572654    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572657    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572659    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572663    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572665    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572668    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572670    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572673    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572676    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 15:48:55.576885 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572678    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572681    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572683    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572686    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572689    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572691    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572694    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572698    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572702    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572705    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572709    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572713    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572717    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572720    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572722    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572725    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572727    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572730    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572732    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 15:48:55.577395 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572735    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572737    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572740    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572742    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572744    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572747    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572749    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572752    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.572754    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572821    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572828    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572835    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572843    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572849    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572854    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572860    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572866    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572870    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572873    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572876    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572881    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572885    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572887    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 23 15:48:55.577878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572890    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572893    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572897    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572900    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572904    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572907    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572910    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572913    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572917    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572921    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572924    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572927    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572930    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572933    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572936    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572939    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572943    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572945    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572950    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572953    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572956    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572960    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572963    2562 flags.go:64] FLAG: --enable-server="true"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572966    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572970    2562 flags.go:64] FLAG: --event-burst="100"
Jul 23 15:48:55.578436 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572973    2562 flags.go:64] FLAG: --event-qps="50"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572976    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572979    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572982    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572987    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572990    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572993    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572996    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.572999    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573002    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573005    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573008    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573011    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573014    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573017    2562 flags.go:64] FLAG: --feature-gates=""
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573020    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573024    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573027    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573030    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573033    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573036    2562 flags.go:64] FLAG: --help="false"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573040    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-140-230.ec2.internal"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573043    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573046    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 15:48:55.579090 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573049    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573052    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573055    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573058    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573063    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573066    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573069    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573072    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573075    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573078    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573081    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573084    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573087    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573090    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573093    2562 flags.go:64] FLAG: --lock-file=""
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573095    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573098    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573101    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573110    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573114    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573117    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573120    2562 flags.go:64] FLAG: --logging-format="text"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573123    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573126    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 15:48:55.579766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573129    2562 flags.go:64] FLAG: --manifest-url=""
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573132    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573136    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573139    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573144    2562 flags.go:64] FLAG: --max-pods="110"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573147    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573150    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573153    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573156    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573159    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573162    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573165    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573173    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573177    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573180    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573183    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573185    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573191    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573194    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573197    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573200    2562 flags.go:64] FLAG: --port="10250"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573203    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573205    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02c65c83cbd474c00"
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573209    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 23 15:48:55.580368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573211    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573215    2562 flags.go:64] FLAG: --register-node="true"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573217    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573221    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573225    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573227    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573230    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573234    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573237    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573240    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573243    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573246    2562 flags.go:64] FLAG: --runonce="false"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573248    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573252    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573254    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573258    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573260    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573263    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573266    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573269    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573272    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573277    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573280    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573282    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573285    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 15:48:55.580978 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573288    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573291    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573296    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573299    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573302    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573307    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573309    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573313    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573316    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573319    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573322    2562 flags.go:64] FLAG: --v="2"
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573326    2562 flags.go:64] FLAG: --version="false"
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573330    2562 flags.go:64] FLAG: --vmodule=""
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573334    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.573337    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573452    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573456    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573473    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573476    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573480    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573484    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573487    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573491    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 15:48:55.581613 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573495    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573498    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573501    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573504    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573507    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573511    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573514    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573516    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573519    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573522    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573525    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573528    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573530    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573533    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573536    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573538    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573541    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573544    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573546    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 15:48:55.582223 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573549    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573552    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573554    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573557    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573560    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573563    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573565    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573568    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573570    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573573    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573577    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573580    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573582    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573585    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573587    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573590    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573592    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573595    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573599    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 15:48:55.582801 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573602    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573604    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573609    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573612    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573615    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573618    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573620    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573623    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573626    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573629    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573632    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573635    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573637    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573640    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573642    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573645    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573648    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573650    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573653    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573656    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 15:48:55.583263 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573658    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573661    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573663    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573670    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573673    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573675    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573678    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573680    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573683    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573686    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573689    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573693    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573696    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573698    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573701    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573704    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573707    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573709    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573712    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573714    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 15:48:55.583786 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573717    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573720    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573723    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573725    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573735    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573738    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573741    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573743    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573746    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573748    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573752    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573755    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.573759    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.574266    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 15:48:55.584288 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.581228    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.581245    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.581265    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.581271    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581332    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581337    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581340    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581343    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581346    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581349    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581352    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581354    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581357    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581360    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581363    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581365    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581368    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581370    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581373    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581376    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 15:48:55.584717 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581378    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581381    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581383    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581386    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581388    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581391    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581393    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581396    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581399    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581402    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581405    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581409    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581414    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581418    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581421    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581424    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581427    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581430    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581433    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581436    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 15:48:55.585197 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581438    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581441    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581444    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581446    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581449    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581452    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581454    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581457    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581473    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581476    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581478    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581481    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581484    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581486    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581490    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581492    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581495    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581498    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581500    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 15:48:55.585796 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581503    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581505    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581509    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581511    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581515    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581519    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581522    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581526    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581529    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581532    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581534    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581537    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581539    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581542    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581544    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581547    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581550    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581552    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581555    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581557    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 15:48:55.586247 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581560    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581563    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581565    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581568    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581570    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581573    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581575    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581578    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581581    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581583    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581586    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581590    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581593    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581596    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581598    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581605    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581608    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581611    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581615    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 15:48:55.586757 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581618    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581620    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581623    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581626    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581629    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.581635    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581749    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581753    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581756    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581760    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581762    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581765    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581767    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581770    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581773    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 15:48:55.587210 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581776    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581778    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581781    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581783    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581786    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581788    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581791    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581793    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581797    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581800    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581803    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581806    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581809    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581811    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581814    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581819    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581822    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581825    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581828    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 15:48:55.587601 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581830    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581833    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581837    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581840    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581843    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581846    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581848    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581851    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581853    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581856    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581858    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581861    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581864    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581866    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581869    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581872    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581874    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581877    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581879    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 15:48:55.588097 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581882    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581885    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581887    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581890    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581893    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581895    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581897    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581900    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581903    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581906    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581909    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581911    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581914    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581917    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581919    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581922    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581924    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581927    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581929    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 15:48:55.588711 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581932    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581934    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581937    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581939    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581941    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581944    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581946    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581949    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581951    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581954    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581956    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581959    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581961    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581963    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581966    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581969    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581971    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581973    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581976    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581978    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 15:48:55.589362 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581981    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581983    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581986    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581992    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581995    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.581997    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.582000    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.582002    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.582005    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.582007    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.582009    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.582012    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:55.582015    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.582020    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.582133    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 15:48:55.589933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.584186    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 15:48:55.590320 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.585341    2562 server.go:1019] "Starting client certificate rotation"
Jul 23 15:48:55.590320 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.585452    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 15:48:55.590320 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.585503    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 15:48:55.590320 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.587961    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 15:48:55.590320 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.589179    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 15:48:55.598610 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.598594    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 23 15:48:55.600492 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.600473    2562 log.go:25] "Validated CRI v1 image API"
Jul 23 15:48:55.601338 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.601325    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 15:48:55.603856 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.603837    2562 fs.go:135] Filesystem UUIDs: map[1299ac47-8c12-4506-aaef-f90ef163ba4d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d83f293c-6731-4da1-af5c-1ddc4d60f0ac:/dev/nvme0n1p3]
Jul 23 15:48:55.603911 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:55.603856    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 15:48:57.053206 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.053089    2562 manager.go:216] Machine: {Timestamp:2026-07-23 15:48:57.051273098 +0000 UTC m=+0.255640482 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099758 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f597dab6af45aedf987fb85e82fef SystemUUID:ec2f597d-ab6a-f45a-edf9-87fb85e82fef BootID:338782a8-a53f-46b8-bd22-1dd47d9a7fc0 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:46:0c:04:d3:35 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:46:0c:04:d3:35 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:4a:91:d9:c5:5c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 15:48:57.053206 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.053199    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 15:48:57.053328 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.053281    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 15:48:57.055089 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.055069    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 15:48:57.055719 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.055693    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 15:48:57.055878 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.055722    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 15:48:57.056394 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.056385    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 15:48:57.056426 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.056398    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 15:48:57.056426 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.056405    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 15:48:57.056983 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.056973    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 15:48:57.058244 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.058233    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 15:48:57.061942 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.061930    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 15:48:57.062053 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.062044    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 15:48:57.062088 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.062073    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 15:48:57.062088 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.062082    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 15:48:57.062139 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.062097    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 15:48:57.062139 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.062108    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 23 15:48:57.062139 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.062117    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 15:48:57.062484 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.062473    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 15:48:57.062525 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.062496    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 15:48:57.064254 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.064236    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 15:48:57.066021 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.066006    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 15:48:57.066102 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.066036    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 15:48:57.067069 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067058    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 15:48:57.067106 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067073    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 15:48:57.067106 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067080    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 15:48:57.067106 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067085    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 15:48:57.067106 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067103    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 15:48:57.067217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067109    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 15:48:57.067217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067118    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 15:48:57.067217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067123    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 15:48:57.067217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067131    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 15:48:57.067217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067136    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 15:48:57.067217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067149    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 15:48:57.067217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067154    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 15:48:57.067728 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067719    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 15:48:57.067728 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.067727    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 15:48:57.071040 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.071024    2562 server.go:1268] "Started kubelet"
Jul 23 15:48:57.071135 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.071096    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 15:48:57.071203 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.071107    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 15:48:57.071203 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.071179    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 15:48:57.071860 ip-10-0-140-230 systemd[1]: Started Kubernetes Kubelet.
Jul 23 15:48:57.072830 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.072666    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 15:48:57.072922 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.072909    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 15:48:57.076850 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.076826    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 15:48:57.079215 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.079188    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 15:48:57.079362 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.079344    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 15:48:57.079477 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.079461    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 15:48:57.079921 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.079907    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 15:48:57.079961 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.079938    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 15:48:57.080082 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.080063    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:57.080690 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.080670    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 15:48:57.080690 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.080691    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 15:48:57.080849 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.080690    2562 factory.go:55] Registering systemd factory
Jul 23 15:48:57.080849 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.080667    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 15:48:57.080849 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.080728    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 23 15:48:57.080849 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.080815    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 15:48:57.080849 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.080822    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 15:48:57.081079 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.079887    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-230.ec2.internal.18c4f5e9d9832e1f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-230.ec2.internal,UID:ip-10-0-140-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-230.ec2.internal,},FirstTimestamp:2026-07-23 15:48:57.070997023 +0000 UTC m=+0.275364407,LastTimestamp:2026-07-23 15:48:57.070997023 +0000 UTC m=+0.275364407,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-230.ec2.internal,}"
Jul 23 15:48:57.081079 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.081059    2562 factory.go:153] Registering CRI-O factory
Jul 23 15:48:57.081185 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.081081    2562 factory.go:223] Registration of the crio container factory successfully
Jul 23 15:48:57.081185 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.081126    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 15:48:57.081185 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.081149    2562 factory.go:103] Registering Raw factory
Jul 23 15:48:57.081185 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.081163    2562 manager.go:1196] Started watching for new ooms in manager
Jul 23 15:48:57.081786 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.081771    2562 manager.go:349] Starting recovery of all containers
Jul 23 15:48:57.083666 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.083626    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 15:48:57.083759 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.083737    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 15:48:57.084661 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.084637    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lk4h4"
Jul 23 15:48:57.092010 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.091994    2562 manager.go:354] Recovery completed
Jul 23 15:48:57.092426 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.092404    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lk4h4"
Jul 23 15:48:57.096594 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.096482    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 15:48:57.098474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.098455    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 15:48:57.098538 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.098486    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 15:48:57.098538 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.098498    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Jul 23 15:48:57.099661 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.099641    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 15:48:57.099661 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.099660    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 15:48:57.099789 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.099678    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 15:48:57.099789 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.099754    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 15:48:57.100876 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.100854    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 15:48:57.101113 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.101099    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 15:48:57.101157 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.101117    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Jul 23 15:48:57.101694 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.101680    2562 policy_none.go:49] "None policy: Start"
Jul 23 15:48:57.101735 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.101638    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-230.ec2.internal.18c4f5e9db266ac1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-230.ec2.internal,UID:ip-10-0-140-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-230.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-230.ec2.internal,},FirstTimestamp:2026-07-23 15:48:57.098472129 +0000 UTC m=+0.302839516,LastTimestamp:2026-07-23 15:48:57.098472129 +0000 UTC m=+0.302839516,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-230.ec2.internal,}"
Jul 23 15:48:57.101790 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.101701    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 15:48:57.101842 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.101818    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 15:48:57.103841 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.103829    2562 policy_none.go:47] "Start"
Jul 23 15:48:57.141404 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.141384    2562 manager.go:337] "Starting Device Plugin manager"
Jul 23 15:48:57.141503 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.141420    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 15:48:57.141503 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.141432    2562 server.go:87] "Starting device plugin registration server"
Jul 23 15:48:57.141684 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.141671    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 15:48:57.141727 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.141686    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 15:48:57.141850 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.141826    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 15:48:57.141966 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.141911    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 15:48:57.141966 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.141920    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 15:48:57.142469 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.142450    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 15:48:57.142569 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.142504    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:57.242245 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.242212    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 15:48:57.244445 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.243208    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 15:48:57.244445 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.243238    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 15:48:57.244445 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.243253    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Jul 23 15:48:57.244445 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.243286    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.246214 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.246191    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 15:48:57.247329 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.247307    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 15:48:57.247431 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.247354    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 15:48:57.247431 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.247374    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 15:48:57.247431 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.247406    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 15:48:57.249857 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.249838    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.249934 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.249866    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-230.ec2.internal\": node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:57.250401 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.250388    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 15:48:57.293176 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.293114    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:57.347664 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.347617    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"]
Jul 23 15:48:57.347820 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.347711    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 15:48:57.348722 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.348705    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 15:48:57.348834 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.348737    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 15:48:57.348834 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.348749    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Jul 23 15:48:57.351068 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.351055    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 15:48:57.351170 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.351157    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.351207 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.351187    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 15:48:57.351813 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.351786    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 15:48:57.351874 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.351829    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 15:48:57.351874 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.351850    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 15:48:57.351874 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.351862    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Jul 23 15:48:57.352004 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.351830    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 15:48:57.352004 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.351929    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Jul 23 15:48:57.354115 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.354101    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.354193 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.354127    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 15:48:57.354823 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.354793    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 15:48:57.354928 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.354837    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 15:48:57.354928 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.354852    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Jul 23 15:48:57.368359 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.368340    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-230.ec2.internal\" not found" node="ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.371876 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.371861    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-230.ec2.internal\" not found" node="ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.382267 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.382249    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ce8e4ca17b6704615fe154a02804011-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"9ce8e4ca17b6704615fe154a02804011\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.382375 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.382272    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ce8e4ca17b6704615fe154a02804011-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"9ce8e4ca17b6704615fe154a02804011\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.382375 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.382288    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ce049313ba5611f719bb6a5b420aca48-config\") pod \"kube-apiserver-proxy-ip-10-0-140-230.ec2.internal\" (UID: \"ce049313ba5611f719bb6a5b420aca48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.393458 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.393437    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:57.483138 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.483098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ce8e4ca17b6704615fe154a02804011-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"9ce8e4ca17b6704615fe154a02804011\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.483138 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.483051    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ce8e4ca17b6704615fe154a02804011-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"9ce8e4ca17b6704615fe154a02804011\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.483293 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.483165    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ce8e4ca17b6704615fe154a02804011-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"9ce8e4ca17b6704615fe154a02804011\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.483293 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.483181    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ce049313ba5611f719bb6a5b420aca48-config\") pod \"kube-apiserver-proxy-ip-10-0-140-230.ec2.internal\" (UID: \"ce049313ba5611f719bb6a5b420aca48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.483293 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.483209    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ce049313ba5611f719bb6a5b420aca48-config\") pod \"kube-apiserver-proxy-ip-10-0-140-230.ec2.internal\" (UID: \"ce049313ba5611f719bb6a5b420aca48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.483293 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.483215    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ce8e4ca17b6704615fe154a02804011-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"9ce8e4ca17b6704615fe154a02804011\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.494190 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.494167    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:57.595035 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.594957    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:57.671194 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.671158    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.674558 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:57.674539    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:57.695081 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.695061    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:57.795699 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.795658    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:57.896206 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.896142    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:57.996737 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:57.996705    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:58.028889 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.028861    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 15:48:58.029498 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.029013    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 15:48:58.097022 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:58.096980    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:58.097022 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.096991    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 15:43:57 +0000 UTC" deadline="2028-03-07 13:55:29.182230043 +0000 UTC"
Jul 23 15:48:58.097022 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.097018    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14230h6m31.085215562s"
Jul 23 15:48:58.197172 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:58.197103    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:58.230208 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.230175    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 15:48:58.253052 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:58.253003    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce049313ba5611f719bb6a5b420aca48.slice/crio-e18fad8672db041f671f5841bb365c4f9afe51b8721b86000dc0bf7b1bb0c921 WatchSource:0}: Error finding container e18fad8672db041f671f5841bb365c4f9afe51b8721b86000dc0bf7b1bb0c921: Status 404 returned error can't find the container with id e18fad8672db041f671f5841bb365c4f9afe51b8721b86000dc0bf7b1bb0c921
Jul 23 15:48:58.253555 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:48:58.253533    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce8e4ca17b6704615fe154a02804011.slice/crio-84ea70fe535dac16b4d5a1b547197d6f46c8a5a078f59b7a05488e203a5fac1c WatchSource:0}: Error finding container 84ea70fe535dac16b4d5a1b547197d6f46c8a5a078f59b7a05488e203a5fac1c: Status 404 returned error can't find the container with id 84ea70fe535dac16b4d5a1b547197d6f46c8a5a078f59b7a05488e203a5fac1c
Jul 23 15:48:58.257613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.257598    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 15:48:58.297768 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:58.297730    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Jul 23 15:48:58.381991 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.381971    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 15:48:58.440751 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.440719    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 15:48:58.481140 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.481063    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:58.493772 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.493751    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 15:48:58.494538 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.494525    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Jul 23 15:48:58.503202 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:58.503182    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 15:48:59.064102 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.064070    2562 apiserver.go:52] "Watching apiserver"
Jul 23 15:48:59.073728 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.073696    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 15:48:59.074296 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.074268    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-zhhqf","openshift-network-operator/iptables-alerter-jq948","kube-system/global-pull-secret-syncer-sl96g","kube-system/konnectivity-agent-99t5l","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r","openshift-cluster-node-tuning-operator/tuned-9cc8m","openshift-dns/node-resolver-564fg","openshift-multus/multus-57vf5","openshift-multus/multus-additional-cni-plugins-97djp","openshift-ovn-kubernetes/ovnkube-node-9x5jr","kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal","openshift-image-registry/node-ca-p995g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal","openshift-multus/network-metrics-daemon-cnjpz"]
Jul 23 15:48:59.077031 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.077005    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-564fg"
Jul 23 15:48:59.079331 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.079310    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.080167 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.080147    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 15:48:59.080514 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.080490    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9ps7f\""
Jul 23 15:48:59.080645 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.080629    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 15:48:59.083747 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.083094    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ck25d\""
Jul 23 15:48:59.083747 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.083181    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 15:48:59.083747 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.083293    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 15:48:59.083747 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.083341    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 15:48:59.083747 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.083474    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:48:59.083747 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.083488    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 15:48:59.083747 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.083539    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:48:59.083747 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.083601    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 15:48:59.084177 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.083841    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:48:59.086166 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.085838    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.088381 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.087199    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 15:48:59.088381 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.087387    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 15:48:59.088830 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.088657    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.089883 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.089497    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v4qlf\""
Jul 23 15:48:59.089883 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.089701    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 15:48:59.090151 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.090135    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 15:48:59.090465 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.090449    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 15:48:59.090755 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.090739    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hsldd\""
Jul 23 15:48:59.091103 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091087    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.091472 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091447    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:48:59.091556 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091491    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cae772ab-5fcb-444a-b8c9-79e8048348e1-hosts-file\") pod \"node-resolver-564fg\" (UID: \"cae772ab-5fcb-444a-b8c9-79e8048348e1\") " pod="openshift-dns/node-resolver-564fg"
Jul 23 15:48:59.091556 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091518    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cae772ab-5fcb-444a-b8c9-79e8048348e1-tmp-dir\") pod \"node-resolver-564fg\" (UID: \"cae772ab-5fcb-444a-b8c9-79e8048348e1\") " pod="openshift-dns/node-resolver-564fg"
Jul 23 15:48:59.091556 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091545    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-socket-dir-parent\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.091710 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091570    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-hostroot\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.091710 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-conf-dir\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.091710 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091631    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9b20f8df-5ab7-4534-8ae3-f51228f128b8-kubelet-config\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:48:59.091710 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091675    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb9s9\" (UniqueName: \"kubernetes.io/projected/cae772ab-5fcb-444a-b8c9-79e8048348e1-kube-api-access-sb9s9\") pod \"node-resolver-564fg\" (UID: \"cae772ab-5fcb-444a-b8c9-79e8048348e1\") " pod="openshift-dns/node-resolver-564fg"
Jul 23 15:48:59.091933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-system-cni-dir\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.091933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-os-release\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.091933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091767    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-run-netns\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.091933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-var-lib-cni-bin\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.091933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091845    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9b20f8df-5ab7-4534-8ae3-f51228f128b8-dbus\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:48:59.091933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad0c93e9-d754-43b4-9310-125f0fe25a9e-cni-binary-copy\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.091933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091895    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-run-k8s-cni-cncf-io\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.091933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-var-lib-kubelet\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.092312 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-run-multus-certs\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.092312 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.091967    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-etc-kubernetes\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.092312 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.092003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbmjl\" (UniqueName: \"kubernetes.io/projected/ad0c93e9-d754-43b4-9310-125f0fe25a9e-kube-api-access-xbmjl\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.092312 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.092084    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-cni-dir\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.092312 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.092110    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-cnibin\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.092312 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.092149    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-var-lib-cni-multus\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.092312 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.092181    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-daemon-config\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.092655 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.092373    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 15:48:59.092655 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.092525    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 15:48:59.092655 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.092596    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kjwcc\""
Jul 23 15:48:59.096969 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.093782    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 15:48:59.096969 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.094307    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:48:59.096969 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.094484    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:48:59.096969 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.095457    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ls6rf\""
Jul 23 15:48:59.097212 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.097114    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 15:48:59.098281 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.098259    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jq948"
Jul 23 15:48:59.100875 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.100854    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 15:48:59.102203 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.102184    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.104826 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.104792    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 15:48:59.105027 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.105011    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 15:48:59.105242 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.105227    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 15:48:59.105429 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.105415    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8mgcn\""
Jul 23 15:48:59.106529 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.106507    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 15:48:59.107248 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.107224    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:48:59.107336 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.107308    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:48:59.107827 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.107788    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 15:48:59.108100 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.108080    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 15:48:59.108303 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.108283    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 15:48:59.108409 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.108389    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 15:48:59.108538 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.108521    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vn86t\""
Jul 23 15:48:59.108601 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.108593    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 15:48:59.109774 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.109752    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p995g"
Jul 23 15:48:59.112303 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.112286    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 15:48:59.112702 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.112682    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 15:48:59.112860 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.112619    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zqw9x\""
Jul 23 15:48:59.112860 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.112578    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 15:48:59.120854 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.120828    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lj8m9"
Jul 23 15:48:59.128702 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.128688    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lj8m9"
Jul 23 15:48:59.182187 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.182163    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 15:48:59.193071 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193040    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1a0d25d-5f09-4812-9861-7678dc1af972-serviceca\") pod \"node-ca-p995g\" (UID: \"f1a0d25d-5f09-4812-9861-7678dc1af972\") " pod="openshift-image-registry/node-ca-p995g"
Jul 23 15:48:59.193241 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193089    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mrb5\" (UniqueName: \"kubernetes.io/projected/f1a0d25d-5f09-4812-9861-7678dc1af972-kube-api-access-9mrb5\") pod \"node-ca-p995g\" (UID: \"f1a0d25d-5f09-4812-9861-7678dc1af972\") " pod="openshift-image-registry/node-ca-p995g"
Jul 23 15:48:59.193241 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193114    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-sysctl-conf\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.193241 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193139    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-run\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.193241 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193169    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb9s9\" (UniqueName: \"kubernetes.io/projected/cae772ab-5fcb-444a-b8c9-79e8048348e1-kube-api-access-sb9s9\") pod \"node-resolver-564fg\" (UID: \"cae772ab-5fcb-444a-b8c9-79e8048348e1\") " pod="openshift-dns/node-resolver-564fg"
Jul 23 15:48:59.193241 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9b20f8df-5ab7-4534-8ae3-f51228f128b8-dbus\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:48:59.193467 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193244    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d451b749-46af-4de3-9409-ad4978d400ee-agent-certs\") pod \"konnectivity-agent-99t5l\" (UID: \"d451b749-46af-4de3-9409-ad4978d400ee\") " pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:48:59.193467 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpv4f\" (UniqueName: \"kubernetes.io/projected/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-kube-api-access-gpv4f\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:48:59.193467 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-modprobe-d\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.193467 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193337    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-systemd\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.193467 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-run-openvswitch\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.193467 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9b20f8df-5ab7-4534-8ae3-f51228f128b8-dbus\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:48:59.193467 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193427    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-etc-kubernetes\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.193467 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193455    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/07e7e5bd-7e31-41af-8981-875038b497bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193477    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-etc-kubernetes\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/07e7e5bd-7e31-41af-8981-875038b497bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193523    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-sys-fs\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193546    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-sysctl-d\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193571    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/578de781-ab34-4bfe-b94a-fb313157b5d2-tmp\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-run-netns\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193625    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-cni-dir\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1a0d25d-5f09-4812-9861-7678dc1af972-host\") pod \"node-ca-p995g\" (UID: \"f1a0d25d-5f09-4812-9861-7678dc1af972\") " pod="openshift-image-registry/node-ca-p995g"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4df03883-9eeb-4b8b-9a6f-556bdcba6eb2-host-slash\") pod \"iptables-alerter-jq948\" (UID: \"4df03883-9eeb-4b8b-9a6f-556bdcba6eb2\") " pod="openshift-network-operator/iptables-alerter-jq948"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193691    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-run-ovn-kubernetes\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193716    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-hostroot\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-cni-dir\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-conf-dir\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193786    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-conf-dir\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193815    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-hostroot\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.193852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-systemd-units\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193863    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-run-systemd\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193887    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-var-lib-openvswitch\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193912    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-run-ovn\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193934    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-log-socket\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193956    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d14e04e1-071c-4b0e-8a00-188ba9797d1a-ovnkube-config\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.193982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-os-release\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-system-cni-dir\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194029    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-registration-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv4t9\" (UniqueName: \"kubernetes.io/projected/65f83d8a-4cf8-4682-be48-d0274b9d0670-kube-api-access-pv4t9\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-os-release\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-lib-modules\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-run-multus-certs\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-sysconfig\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194143    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-cnibin\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-var-lib-cni-multus\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194190    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-daemon-config\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.194603 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d14e04e1-071c-4b0e-8a00-188ba9797d1a-ovnkube-script-lib\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cae772ab-5fcb-444a-b8c9-79e8048348e1-tmp-dir\") pod \"node-resolver-564fg\" (UID: \"cae772ab-5fcb-444a-b8c9-79e8048348e1\") " pod="openshift-dns/node-resolver-564fg"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-socket-dir-parent\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-cnibin\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194463    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kqlx\" (UniqueName: \"kubernetes.io/projected/578de781-ab34-4bfe-b94a-fb313157b5d2-kube-api-access-5kqlx\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194496    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-run-netns\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194539    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194588    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-run-netns\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194619    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-etc-selinux\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194644    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-tuned\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194649    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-run-multus-certs\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194653    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-var-lib-cni-multus\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194670    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zxhs\" (UniqueName: \"kubernetes.io/projected/4df03883-9eeb-4b8b-9a6f-556bdcba6eb2-kube-api-access-7zxhs\") pod \"iptables-alerter-jq948\" (UID: \"4df03883-9eeb-4b8b-9a6f-556bdcba6eb2\") " pod="openshift-network-operator/iptables-alerter-jq948"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194699    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcvs6\" (UniqueName: \"kubernetes.io/projected/d14e04e1-071c-4b0e-8a00-188ba9797d1a-kube-api-access-vcvs6\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194716    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-socket-dir-parent\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194738    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-run-k8s-cni-cncf-io\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-var-lib-kubelet\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.195450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194827    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbmjl\" (UniqueName: \"kubernetes.io/projected/ad0c93e9-d754-43b4-9310-125f0fe25a9e-kube-api-access-xbmjl\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194839    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-var-lib-kubelet\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194861    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8dxr\" (UniqueName: \"kubernetes.io/projected/07e7e5bd-7e31-41af-8981-875038b497bc-kube-api-access-b8dxr\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194779    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-run-k8s-cni-cncf-io\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194888    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-socket-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-device-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194934    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-var-lib-kubelet\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194978    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/07e7e5bd-7e31-41af-8981-875038b497bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.194992    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ad0c93e9-d754-43b4-9310-125f0fe25a9e-multus-daemon-config\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195001    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-sys\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-host\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195029    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cae772ab-5fcb-444a-b8c9-79e8048348e1-tmp-dir\") pod \"node-resolver-564fg\" (UID: \"cae772ab-5fcb-444a-b8c9-79e8048348e1\") " pod="openshift-dns/node-resolver-564fg"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195048    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-kubelet\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195092    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-slash\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195117    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-etc-openvswitch\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.195151    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 15:48:59.196330 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195161    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.195235    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret podName:9b20f8df-5ab7-4534-8ae3-f51228f128b8 nodeName:}" failed. No retries permitted until 2026-07-23 15:48:59.695215115 +0000 UTC m=+2.899582490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret") pod "global-pull-secret-syncer-sl96g" (UID: "9b20f8df-5ab7-4534-8ae3-f51228f128b8") : object "kube-system"/"original-pull-secret" not registered
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195268    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cae772ab-5fcb-444a-b8c9-79e8048348e1-hosts-file\") pod \"node-resolver-564fg\" (UID: \"cae772ab-5fcb-444a-b8c9-79e8048348e1\") " pod="openshift-dns/node-resolver-564fg"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195303    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9b20f8df-5ab7-4534-8ae3-f51228f128b8-kubelet-config\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195357    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9b20f8df-5ab7-4534-8ae3-f51228f128b8-kubelet-config\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195387    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-os-release\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195398    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cae772ab-5fcb-444a-b8c9-79e8048348e1-hosts-file\") pod \"node-resolver-564fg\" (UID: \"cae772ab-5fcb-444a-b8c9-79e8048348e1\") " pod="openshift-dns/node-resolver-564fg"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195423    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195453    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-cni-bin\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195479    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d14e04e1-071c-4b0e-8a00-188ba9797d1a-env-overrides\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-system-cni-dir\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195559    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-var-lib-cni-bin\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195588    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwmmj\" (UniqueName: \"kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj\") pod \"network-check-target-zhhqf\" (UID: \"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441\") " pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195615    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-kubernetes\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195621    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-host-var-lib-cni-bin\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad0c93e9-d754-43b4-9310-125f0fe25a9e-system-cni-dir\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.197162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195644    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-node-log\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.197973 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad0c93e9-d754-43b4-9310-125f0fe25a9e-cni-binary-copy\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.197973 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-cnibin\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.197973 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195756    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-kubelet-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.197973 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195785    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4df03883-9eeb-4b8b-9a6f-556bdcba6eb2-iptables-alerter-script\") pod \"iptables-alerter-jq948\" (UID: \"4df03883-9eeb-4b8b-9a6f-556bdcba6eb2\") " pod="openshift-network-operator/iptables-alerter-jq948"
Jul 23 15:48:59.197973 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d451b749-46af-4de3-9409-ad4978d400ee-konnectivity-ca\") pod \"konnectivity-agent-99t5l\" (UID: \"d451b749-46af-4de3-9409-ad4978d400ee\") " pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:48:59.197973 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195859    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-cni-netd\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.197973 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.195888    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d14e04e1-071c-4b0e-8a00-188ba9797d1a-ovn-node-metrics-cert\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.197973 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.196150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad0c93e9-d754-43b4-9310-125f0fe25a9e-cni-binary-copy\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.200912 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.200879    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 15:48:59.205153 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.205085    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb9s9\" (UniqueName: \"kubernetes.io/projected/cae772ab-5fcb-444a-b8c9-79e8048348e1-kube-api-access-sb9s9\") pod \"node-resolver-564fg\" (UID: \"cae772ab-5fcb-444a-b8c9-79e8048348e1\") " pod="openshift-dns/node-resolver-564fg"
Jul 23 15:48:59.206410 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.206385    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbmjl\" (UniqueName: \"kubernetes.io/projected/ad0c93e9-d754-43b4-9310-125f0fe25a9e-kube-api-access-xbmjl\") pod \"multus-57vf5\" (UID: \"ad0c93e9-d754-43b4-9310-125f0fe25a9e\") " pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.253329 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.253276    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal" event={"ID":"9ce8e4ca17b6704615fe154a02804011","Type":"ContainerStarted","Data":"84ea70fe535dac16b4d5a1b547197d6f46c8a5a078f59b7a05488e203a5fac1c"}
Jul 23 15:48:59.254907 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.254881    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal" event={"ID":"ce049313ba5611f719bb6a5b420aca48","Type":"ContainerStarted","Data":"e18fad8672db041f671f5841bb365c4f9afe51b8721b86000dc0bf7b1bb0c921"}
Jul 23 15:48:59.296461 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296423    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.296646 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296471    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-etc-selinux\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.296646 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296501    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-tuned\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.296646 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296525    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zxhs\" (UniqueName: \"kubernetes.io/projected/4df03883-9eeb-4b8b-9a6f-556bdcba6eb2-kube-api-access-7zxhs\") pod \"iptables-alerter-jq948\" (UID: \"4df03883-9eeb-4b8b-9a6f-556bdcba6eb2\") " pod="openshift-network-operator/iptables-alerter-jq948"
Jul 23 15:48:59.296646 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcvs6\" (UniqueName: \"kubernetes.io/projected/d14e04e1-071c-4b0e-8a00-188ba9797d1a-kube-api-access-vcvs6\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.296646 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8dxr\" (UniqueName: \"kubernetes.io/projected/07e7e5bd-7e31-41af-8981-875038b497bc-kube-api-access-b8dxr\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.296646 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296605    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-socket-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.296646 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-device-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.296646 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296650    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-var-lib-kubelet\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296687    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/07e7e5bd-7e31-41af-8981-875038b497bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-sys\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296773    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-host\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296789    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-device-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296812    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-kubelet\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-var-lib-kubelet\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296853    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-slash\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296876    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-etc-openvswitch\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296924    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-os-release\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296937    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-socket-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296958    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.296982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-cni-bin\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297005    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d14e04e1-071c-4b0e-8a00-188ba9797d1a-env-overrides\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwmmj\" (UniqueName: \"kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj\") pod \"network-check-target-zhhqf\" (UID: \"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441\") " pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:48:59.297082 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297082    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-kubernetes\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297106    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-node-log\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297131    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-cnibin\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297133    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-etc-selinux\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297191    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-kubelet-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4df03883-9eeb-4b8b-9a6f-556bdcba6eb2-iptables-alerter-script\") pod \"iptables-alerter-jq948\" (UID: \"4df03883-9eeb-4b8b-9a6f-556bdcba6eb2\") " pod="openshift-network-operator/iptables-alerter-jq948"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297261    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297273    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d451b749-46af-4de3-9409-ad4978d400ee-konnectivity-ca\") pod \"konnectivity-agent-99t5l\" (UID: \"d451b749-46af-4de3-9409-ad4978d400ee\") " pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297312    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-kubelet\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297349    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-slash\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297364    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-host\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297374    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-etc-openvswitch\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297397    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-os-release\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297410    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-kubernetes\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297419    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-kubelet-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297445    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-sys\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.297839 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297453    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-cni-netd\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297463    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-cni-bin\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d14e04e1-071c-4b0e-8a00-188ba9797d1a-ovn-node-metrics-cert\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297502    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-cnibin\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1a0d25d-5f09-4812-9861-7678dc1af972-serviceca\") pod \"node-ca-p995g\" (UID: \"f1a0d25d-5f09-4812-9861-7678dc1af972\") " pod="openshift-image-registry/node-ca-p995g"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297537    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mrb5\" (UniqueName: \"kubernetes.io/projected/f1a0d25d-5f09-4812-9861-7678dc1af972-kube-api-access-9mrb5\") pod \"node-ca-p995g\" (UID: \"f1a0d25d-5f09-4812-9861-7678dc1af972\") " pod="openshift-image-registry/node-ca-p995g"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297544    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-cni-netd\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297554    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-node-log\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297564    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-sysctl-conf\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-run\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297641    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d451b749-46af-4de3-9409-ad4978d400ee-agent-certs\") pod \"konnectivity-agent-99t5l\" (UID: \"d451b749-46af-4de3-9409-ad4978d400ee\") " pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297665    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpv4f\" (UniqueName: \"kubernetes.io/projected/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-kube-api-access-gpv4f\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-modprobe-d\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297701    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-systemd\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297727    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-run-openvswitch\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297748    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/07e7e5bd-7e31-41af-8981-875038b497bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297754    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/07e7e5bd-7e31-41af-8981-875038b497bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.298604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/07e7e5bd-7e31-41af-8981-875038b497bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297793    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-sys-fs\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297831    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-sysctl-d\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/578de781-ab34-4bfe-b94a-fb313157b5d2-tmp\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297879    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-run-netns\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297903    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1a0d25d-5f09-4812-9861-7678dc1af972-host\") pod \"node-ca-p995g\" (UID: \"f1a0d25d-5f09-4812-9861-7678dc1af972\") " pod="openshift-image-registry/node-ca-p995g"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297926    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4df03883-9eeb-4b8b-9a6f-556bdcba6eb2-host-slash\") pod \"iptables-alerter-jq948\" (UID: \"4df03883-9eeb-4b8b-9a6f-556bdcba6eb2\") " pod="openshift-network-operator/iptables-alerter-jq948"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297950    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-run-ovn-kubernetes\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.297976    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-systemd-units\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4df03883-9eeb-4b8b-9a6f-556bdcba6eb2-iptables-alerter-script\") pod \"iptables-alerter-jq948\" (UID: \"4df03883-9eeb-4b8b-9a6f-556bdcba6eb2\") " pod="openshift-network-operator/iptables-alerter-jq948"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298015    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-run-systemd\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298021    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-systemd-units\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-var-lib-openvswitch\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298116    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1a0d25d-5f09-4812-9861-7678dc1af972-serviceca\") pod \"node-ca-p995g\" (UID: \"f1a0d25d-5f09-4812-9861-7678dc1af972\") " pod="openshift-image-registry/node-ca-p995g"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-systemd\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-run-systemd\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d451b749-46af-4de3-9409-ad4978d400ee-konnectivity-ca\") pod \"konnectivity-agent-99t5l\" (UID: \"d451b749-46af-4de3-9409-ad4978d400ee\") " pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298246    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-sys-fs\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.299474 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.298254    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298299    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-sysctl-conf\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.298314    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs podName:222a0a15-7ded-4543-94e0-21ddcd1bcf3d nodeName:}" failed. No retries permitted until 2026-07-23 15:48:59.79829596 +0000 UTC m=+3.002663344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs") pod "network-metrics-daemon-cnjpz" (UID: "222a0a15-7ded-4543-94e0-21ddcd1bcf3d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298398    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-modprobe-d\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298400    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d14e04e1-071c-4b0e-8a00-188ba9797d1a-env-overrides\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298430    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-var-lib-openvswitch\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-run-ovn\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298458    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-run-netns\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298461    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-log-socket\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298473    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-run\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298495    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-log-socket\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d14e04e1-071c-4b0e-8a00-188ba9797d1a-ovnkube-config\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298511    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-run-ovn\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298550    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-run-openvswitch\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298580    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-system-cni-dir\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-registration-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298639    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv4t9\" (UniqueName: \"kubernetes.io/projected/65f83d8a-4cf8-4682-be48-d0274b9d0670-kube-api-access-pv4t9\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.300337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-lib-modules\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-sysconfig\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298737    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d14e04e1-071c-4b0e-8a00-188ba9797d1a-ovnkube-script-lib\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298966    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/07e7e5bd-7e31-41af-8981-875038b497bc-system-cni-dir\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.299007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kqlx\" (UniqueName: \"kubernetes.io/projected/578de781-ab34-4bfe-b94a-fb313157b5d2-kube-api-access-5kqlx\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.298965    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/07e7e5bd-7e31-41af-8981-875038b497bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.299065    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/07e7e5bd-7e31-41af-8981-875038b497bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.299075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d14e04e1-071c-4b0e-8a00-188ba9797d1a-ovnkube-config\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.299079    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-lib-modules\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.299116    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4df03883-9eeb-4b8b-9a6f-556bdcba6eb2-host-slash\") pod \"iptables-alerter-jq948\" (UID: \"4df03883-9eeb-4b8b-9a6f-556bdcba6eb2\") " pod="openshift-network-operator/iptables-alerter-jq948"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.299134    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-sysconfig\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.299154    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1a0d25d-5f09-4812-9861-7678dc1af972-host\") pod \"node-ca-p995g\" (UID: \"f1a0d25d-5f09-4812-9861-7678dc1af972\") " pod="openshift-image-registry/node-ca-p995g"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.299192    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d14e04e1-071c-4b0e-8a00-188ba9797d1a-host-run-ovn-kubernetes\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.299203    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-sysctl-d\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.299332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/65f83d8a-4cf8-4682-be48-d0274b9d0670-registration-dir\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.301216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.299721    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d14e04e1-071c-4b0e-8a00-188ba9797d1a-ovnkube-script-lib\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.301925 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.301606    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/578de781-ab34-4bfe-b94a-fb313157b5d2-tmp\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.301994 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.301971    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d14e04e1-071c-4b0e-8a00-188ba9797d1a-ovn-node-metrics-cert\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.302303 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.302260    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/578de781-ab34-4bfe-b94a-fb313157b5d2-etc-tuned\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.303692 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.303668    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d451b749-46af-4de3-9409-ad4978d400ee-agent-certs\") pod \"konnectivity-agent-99t5l\" (UID: \"d451b749-46af-4de3-9409-ad4978d400ee\") " pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:48:59.307550 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.306726    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zxhs\" (UniqueName: \"kubernetes.io/projected/4df03883-9eeb-4b8b-9a6f-556bdcba6eb2-kube-api-access-7zxhs\") pod \"iptables-alerter-jq948\" (UID: \"4df03883-9eeb-4b8b-9a6f-556bdcba6eb2\") " pod="openshift-network-operator/iptables-alerter-jq948"
Jul 23 15:48:59.307550 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.306857    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8dxr\" (UniqueName: \"kubernetes.io/projected/07e7e5bd-7e31-41af-8981-875038b497bc-kube-api-access-b8dxr\") pod \"multus-additional-cni-plugins-97djp\" (UID: \"07e7e5bd-7e31-41af-8981-875038b497bc\") " pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.307550 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.307036    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 15:48:59.307550 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.307061    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 15:48:59.307550 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.307075    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wwmmj for pod openshift-network-diagnostics/network-check-target-zhhqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:48:59.307550 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.307136    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj podName:1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441 nodeName:}" failed. No retries permitted until 2026-07-23 15:48:59.807117634 +0000 UTC m=+3.011485008 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wwmmj" (UniqueName: "kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj") pod "network-check-target-zhhqf" (UID: "1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:48:59.308226 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.308180    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcvs6\" (UniqueName: \"kubernetes.io/projected/d14e04e1-071c-4b0e-8a00-188ba9797d1a-kube-api-access-vcvs6\") pod \"ovnkube-node-9x5jr\" (UID: \"d14e04e1-071c-4b0e-8a00-188ba9797d1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.312657 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.312417    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpv4f\" (UniqueName: \"kubernetes.io/projected/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-kube-api-access-gpv4f\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:48:59.312657 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.312444    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mrb5\" (UniqueName: \"kubernetes.io/projected/f1a0d25d-5f09-4812-9861-7678dc1af972-kube-api-access-9mrb5\") pod \"node-ca-p995g\" (UID: \"f1a0d25d-5f09-4812-9861-7678dc1af972\") " pod="openshift-image-registry/node-ca-p995g"
Jul 23 15:48:59.312890 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.312772    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv4t9\" (UniqueName: \"kubernetes.io/projected/65f83d8a-4cf8-4682-be48-d0274b9d0670-kube-api-access-pv4t9\") pod \"aws-ebs-csi-driver-node-98n6r\" (UID: \"65f83d8a-4cf8-4682-be48-d0274b9d0670\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.313944 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.313923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kqlx\" (UniqueName: \"kubernetes.io/projected/578de781-ab34-4bfe-b94a-fb313157b5d2-kube-api-access-5kqlx\") pod \"tuned-9cc8m\" (UID: \"578de781-ab34-4bfe-b94a-fb313157b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.395003 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.394917    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-564fg"
Jul 23 15:48:59.413148 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.413116    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-57vf5"
Jul 23 15:48:59.422298 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.422269    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:48:59.429606 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.429524    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r"
Jul 23 15:48:59.437388 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.437364    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9cc8m"
Jul 23 15:48:59.445273 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.445245    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-97djp"
Jul 23 15:48:59.454883 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.454857    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jq948"
Jul 23 15:48:59.462668 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.462647    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:48:59.469528 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.469511    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p995g"
Jul 23 15:48:59.512774 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.512743    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 15:48:59.701136 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.701045    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:48:59.701304 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.701244    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 15:48:59.701374 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.701325    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret podName:9b20f8df-5ab7-4534-8ae3-f51228f128b8 nodeName:}" failed. No retries permitted until 2026-07-23 15:49:00.701306314 +0000 UTC m=+3.905673685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret") pod "global-pull-secret-syncer-sl96g" (UID: "9b20f8df-5ab7-4534-8ae3-f51228f128b8") : object "kube-system"/"original-pull-secret" not registered
Jul 23 15:48:59.801813 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.801761    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:48:59.802006 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.801928    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:48:59.802006 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.802005    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs podName:222a0a15-7ded-4543-94e0-21ddcd1bcf3d nodeName:}" failed. No retries permitted until 2026-07-23 15:49:00.801985578 +0000 UTC m=+4.006352963 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs") pod "network-metrics-daemon-cnjpz" (UID: "222a0a15-7ded-4543-94e0-21ddcd1bcf3d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:48:59.902791 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:48:59.902752    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwmmj\" (UniqueName: \"kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj\") pod \"network-check-target-zhhqf\" (UID: \"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441\") " pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:48:59.902973 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.902912    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 15:48:59.902973 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.902932    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 15:48:59.902973 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.902943    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wwmmj for pod openshift-network-diagnostics/network-check-target-zhhqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:48:59.903123 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:48:59.903011    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj podName:1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441 nodeName:}" failed. No retries permitted until 2026-07-23 15:49:00.9029877 +0000 UTC m=+4.107355094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwmmj" (UniqueName: "kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj") pod "network-check-target-zhhqf" (UID: "1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:49:00.132164 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:00.132081    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 15:43:59 +0000 UTC" deadline="2028-05-02 14:22:33.217262026 +0000 UTC"
Jul 23 15:49:00.132164 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:00.132111    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15574h33m33.085155598s"
Jul 23 15:49:00.247871 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:00.247833    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:00.247871 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:00.247864    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:00.248087 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:00.247853    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:00.248087 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:00.247951    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:00.248087 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:00.248072    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:00.248202 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:00.248144    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:00.384248 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:00.384169    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae772ab_5fcb_444a_b8c9_79e8048348e1.slice/crio-225dd27e1f5ae7544c1e988f6815b06e84f936c37bdd0b0c166b3fd69d8eab06 WatchSource:0}: Error finding container 225dd27e1f5ae7544c1e988f6815b06e84f936c37bdd0b0c166b3fd69d8eab06: Status 404 returned error can't find the container with id 225dd27e1f5ae7544c1e988f6815b06e84f936c37bdd0b0c166b3fd69d8eab06
Jul 23 15:49:00.388579 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:00.388555    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df03883_9eeb_4b8b_9a6f_556bdcba6eb2.slice/crio-5c1ac48cacd414964b55395d031882137a3f8fc2e41a916d655715bca4b46416 WatchSource:0}: Error finding container 5c1ac48cacd414964b55395d031882137a3f8fc2e41a916d655715bca4b46416: Status 404 returned error can't find the container with id 5c1ac48cacd414964b55395d031882137a3f8fc2e41a916d655715bca4b46416
Jul 23 15:49:00.391159 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:00.391135    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a0d25d_5f09_4812_9861_7678dc1af972.slice/crio-2ff85f4e75677414afe53842632109a6b8791f0047009f2ec504d6e077ba932d WatchSource:0}: Error finding container 2ff85f4e75677414afe53842632109a6b8791f0047009f2ec504d6e077ba932d: Status 404 returned error can't find the container with id 2ff85f4e75677414afe53842632109a6b8791f0047009f2ec504d6e077ba932d
Jul 23 15:49:00.392478 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:00.392399    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod578de781_ab34_4bfe_b94a_fb313157b5d2.slice/crio-172df2cda6f217e71617c5793b3e84bb4b45e1e7b06b9a41756a3a62a8d6ad1f WatchSource:0}: Error finding container 172df2cda6f217e71617c5793b3e84bb4b45e1e7b06b9a41756a3a62a8d6ad1f: Status 404 returned error can't find the container with id 172df2cda6f217e71617c5793b3e84bb4b45e1e7b06b9a41756a3a62a8d6ad1f
Jul 23 15:49:00.393765 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:00.393740    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad0c93e9_d754_43b4_9310_125f0fe25a9e.slice/crio-6f54750f290cdb1aab2e632881f1660016979bf6a2afcb1e76a5f1ffbd81e106 WatchSource:0}: Error finding container 6f54750f290cdb1aab2e632881f1660016979bf6a2afcb1e76a5f1ffbd81e106: Status 404 returned error can't find the container with id 6f54750f290cdb1aab2e632881f1660016979bf6a2afcb1e76a5f1ffbd81e106
Jul 23 15:49:00.394675 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:00.394376    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd451b749_46af_4de3_9409_ad4978d400ee.slice/crio-1378a3d69c8a3711c31c525abfb2a7d0fe6e797dc0206df0d768bd5201de1c17 WatchSource:0}: Error finding container 1378a3d69c8a3711c31c525abfb2a7d0fe6e797dc0206df0d768bd5201de1c17: Status 404 returned error can't find the container with id 1378a3d69c8a3711c31c525abfb2a7d0fe6e797dc0206df0d768bd5201de1c17
Jul 23 15:49:00.395201 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:00.395181    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07e7e5bd_7e31_41af_8981_875038b497bc.slice/crio-da21852e7d219e5335217b319859cba377af211734cf1c12315b529aed6af70e WatchSource:0}: Error finding container da21852e7d219e5335217b319859cba377af211734cf1c12315b529aed6af70e: Status 404 returned error can't find the container with id da21852e7d219e5335217b319859cba377af211734cf1c12315b529aed6af70e
Jul 23 15:49:00.418084 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:00.417770    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14e04e1_071c_4b0e_8a00_188ba9797d1a.slice/crio-d43224f306186df4fc0889155f040d96ba1df5288da5537219f94d4729645e1e WatchSource:0}: Error finding container d43224f306186df4fc0889155f040d96ba1df5288da5537219f94d4729645e1e: Status 404 returned error can't find the container with id d43224f306186df4fc0889155f040d96ba1df5288da5537219f94d4729645e1e
Jul 23 15:49:00.418842 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:00.418818    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f83d8a_4cf8_4682_be48_d0274b9d0670.slice/crio-f3dfdc020129af6882a6361cc5e1d88b0b88453e67af27f3ff04d20e150df1ff WatchSource:0}: Error finding container f3dfdc020129af6882a6361cc5e1d88b0b88453e67af27f3ff04d20e150df1ff: Status 404 returned error can't find the container with id f3dfdc020129af6882a6361cc5e1d88b0b88453e67af27f3ff04d20e150df1ff
Jul 23 15:49:00.708817 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:00.708542    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:00.708817 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:00.708680    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 15:49:00.708989 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:00.708851    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret podName:9b20f8df-5ab7-4534-8ae3-f51228f128b8 nodeName:}" failed. No retries permitted until 2026-07-23 15:49:02.708836901 +0000 UTC m=+5.913204272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret") pod "global-pull-secret-syncer-sl96g" (UID: "9b20f8df-5ab7-4534-8ae3-f51228f128b8") : object "kube-system"/"original-pull-secret" not registered
Jul 23 15:49:00.810046 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:00.810013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:00.810205 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:00.810181    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:49:00.810258 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:00.810245    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs podName:222a0a15-7ded-4543-94e0-21ddcd1bcf3d nodeName:}" failed. No retries permitted until 2026-07-23 15:49:02.810230193 +0000 UTC m=+6.014597573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs") pod "network-metrics-daemon-cnjpz" (UID: "222a0a15-7ded-4543-94e0-21ddcd1bcf3d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:49:00.910659 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:00.910619    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwmmj\" (UniqueName: \"kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj\") pod \"network-check-target-zhhqf\" (UID: \"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441\") " pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:00.910864 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:00.910834    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 15:49:00.910864 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:00.910858    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 15:49:00.910986 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:00.910871    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wwmmj for pod openshift-network-diagnostics/network-check-target-zhhqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:49:00.910986 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:00.910941    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj podName:1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441 nodeName:}" failed. No retries permitted until 2026-07-23 15:49:02.910922375 +0000 UTC m=+6.115289765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwmmj" (UniqueName: "kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj") pod "network-check-target-zhhqf" (UID: "1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:49:01.132721 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.132653    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 15:43:59 +0000 UTC" deadline="2028-01-15 04:57:27.381024103 +0000 UTC"
Jul 23 15:49:01.132721 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.132718    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12973h8m26.24831018s"
Jul 23 15:49:01.263185 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.263101    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal" event={"ID":"ce049313ba5611f719bb6a5b420aca48","Type":"ContainerStarted","Data":"1b25dc5c5efa69b37427ae5a569d376f048b134cf78de2cd6d1522d93c54d039"}
Jul 23 15:49:01.268585 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.268549    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr" event={"ID":"d14e04e1-071c-4b0e-8a00-188ba9797d1a","Type":"ContainerStarted","Data":"d43224f306186df4fc0889155f040d96ba1df5288da5537219f94d4729645e1e"}
Jul 23 15:49:01.277638 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.277604    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r" event={"ID":"65f83d8a-4cf8-4682-be48-d0274b9d0670","Type":"ContainerStarted","Data":"f3dfdc020129af6882a6361cc5e1d88b0b88453e67af27f3ff04d20e150df1ff"}
Jul 23 15:49:01.280162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.280108    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal" podStartSLOduration=3.280088968 podStartE2EDuration="3.280088968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:48:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:01.27895874 +0000 UTC m=+4.483326134" watchObservedRunningTime="2026-07-23 15:49:01.280088968 +0000 UTC m=+4.484456362"
Jul 23 15:49:01.281856 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.281747    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p995g" event={"ID":"f1a0d25d-5f09-4812-9861-7678dc1af972","Type":"ContainerStarted","Data":"2ff85f4e75677414afe53842632109a6b8791f0047009f2ec504d6e077ba932d"}
Jul 23 15:49:01.288279 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.288254    2562 generic.go:356] "Generic (PLEG): container finished" podID="9ce8e4ca17b6704615fe154a02804011" containerID="5bdd347eec01892d55f6a66d536e181047d231a8cc72f92c26e9e427f85f129a" exitCode=0
Jul 23 15:49:01.288390 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.288343    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal" event={"ID":"9ce8e4ca17b6704615fe154a02804011","Type":"ContainerDied","Data":"5bdd347eec01892d55f6a66d536e181047d231a8cc72f92c26e9e427f85f129a"}
Jul 23 15:49:01.290161 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.290136    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97djp" event={"ID":"07e7e5bd-7e31-41af-8981-875038b497bc","Type":"ContainerStarted","Data":"da21852e7d219e5335217b319859cba377af211734cf1c12315b529aed6af70e"}
Jul 23 15:49:01.294380 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.294354    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-99t5l" event={"ID":"d451b749-46af-4de3-9409-ad4978d400ee","Type":"ContainerStarted","Data":"1378a3d69c8a3711c31c525abfb2a7d0fe6e797dc0206df0d768bd5201de1c17"}
Jul 23 15:49:01.297598 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.297570    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-57vf5" event={"ID":"ad0c93e9-d754-43b4-9310-125f0fe25a9e","Type":"ContainerStarted","Data":"6f54750f290cdb1aab2e632881f1660016979bf6a2afcb1e76a5f1ffbd81e106"}
Jul 23 15:49:01.299959 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.299894    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9cc8m" event={"ID":"578de781-ab34-4bfe-b94a-fb313157b5d2","Type":"ContainerStarted","Data":"172df2cda6f217e71617c5793b3e84bb4b45e1e7b06b9a41756a3a62a8d6ad1f"}
Jul 23 15:49:01.314655 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.314623    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jq948" event={"ID":"4df03883-9eeb-4b8b-9a6f-556bdcba6eb2","Type":"ContainerStarted","Data":"5c1ac48cacd414964b55395d031882137a3f8fc2e41a916d655715bca4b46416"}
Jul 23 15:49:01.329374 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:01.329336    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-564fg" event={"ID":"cae772ab-5fcb-444a-b8c9-79e8048348e1","Type":"ContainerStarted","Data":"225dd27e1f5ae7544c1e988f6815b06e84f936c37bdd0b0c166b3fd69d8eab06"}
Jul 23 15:49:02.248015 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:02.247981    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:02.248487 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:02.248240    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:02.248765 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:02.248745    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:02.248892 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:02.248863    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:02.248969 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:02.248955    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:02.249058 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:02.249040    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:02.351306 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:02.351270    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal" event={"ID":"9ce8e4ca17b6704615fe154a02804011","Type":"ContainerStarted","Data":"d1d584fba753ca2d10ecbddb253ce7104648f4cfe8c36fab7fb732ec7785750b"}
Jul 23 15:49:02.368621 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:02.368563    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal" podStartSLOduration=4.368543669 podStartE2EDuration="4.368543669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:48:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:02.367776238 +0000 UTC m=+5.572143632" watchObservedRunningTime="2026-07-23 15:49:02.368543669 +0000 UTC m=+5.572911064"
Jul 23 15:49:02.732568 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:02.732470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:02.732745 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:02.732682    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 15:49:02.732827 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:02.732752    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret podName:9b20f8df-5ab7-4534-8ae3-f51228f128b8 nodeName:}" failed. No retries permitted until 2026-07-23 15:49:06.732733577 +0000 UTC m=+9.937100949 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret") pod "global-pull-secret-syncer-sl96g" (UID: "9b20f8df-5ab7-4534-8ae3-f51228f128b8") : object "kube-system"/"original-pull-secret" not registered
Jul 23 15:49:02.833748 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:02.833710    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:02.833929 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:02.833894    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:49:02.834011 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:02.833977    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs podName:222a0a15-7ded-4543-94e0-21ddcd1bcf3d nodeName:}" failed. No retries permitted until 2026-07-23 15:49:06.833952978 +0000 UTC m=+10.038320357 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs") pod "network-metrics-daemon-cnjpz" (UID: "222a0a15-7ded-4543-94e0-21ddcd1bcf3d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:49:02.935149 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:02.935110    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwmmj\" (UniqueName: \"kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj\") pod \"network-check-target-zhhqf\" (UID: \"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441\") " pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:02.935322 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:02.935308    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 15:49:02.935415 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:02.935335    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 15:49:02.935415 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:02.935348    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wwmmj for pod openshift-network-diagnostics/network-check-target-zhhqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:49:02.935415 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:02.935412    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj podName:1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441 nodeName:}" failed. No retries permitted until 2026-07-23 15:49:06.935393473 +0000 UTC m=+10.139760843 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwmmj" (UniqueName: "kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj") pod "network-check-target-zhhqf" (UID: "1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:49:04.248475 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:04.247999    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:04.248475 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:04.248044    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:04.248475 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:04.248128    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:04.248475 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:04.248185    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:04.248475 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:04.248252    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:04.248475 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:04.248343    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:06.247895 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:06.247842    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:06.247895 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:06.247871    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:06.248419 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:06.247992    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:06.248419 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:06.248070    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:06.248419 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:06.248074    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:06.248419 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:06.248158    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:06.764650 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:06.764596    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:06.764848 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:06.764757    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 15:49:06.764924 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:06.764853    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret podName:9b20f8df-5ab7-4534-8ae3-f51228f128b8 nodeName:}" failed. No retries permitted until 2026-07-23 15:49:14.764829804 +0000 UTC m=+17.969197178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret") pod "global-pull-secret-syncer-sl96g" (UID: "9b20f8df-5ab7-4534-8ae3-f51228f128b8") : object "kube-system"/"original-pull-secret" not registered
Jul 23 15:49:06.865955 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:06.865911    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:06.866127 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:06.866094    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:49:06.866204 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:06.866181    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs podName:222a0a15-7ded-4543-94e0-21ddcd1bcf3d nodeName:}" failed. No retries permitted until 2026-07-23 15:49:14.866158016 +0000 UTC m=+18.070525393 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs") pod "network-metrics-daemon-cnjpz" (UID: "222a0a15-7ded-4543-94e0-21ddcd1bcf3d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:49:06.967331 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:06.966706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwmmj\" (UniqueName: \"kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj\") pod \"network-check-target-zhhqf\" (UID: \"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441\") " pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:06.967331 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:06.966900    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 15:49:06.967331 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:06.966923    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 15:49:06.967331 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:06.966936    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wwmmj for pod openshift-network-diagnostics/network-check-target-zhhqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:49:06.967331 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:06.966996    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj podName:1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441 nodeName:}" failed. No retries permitted until 2026-07-23 15:49:14.966979005 +0000 UTC m=+18.171346379 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwmmj" (UniqueName: "kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj") pod "network-check-target-zhhqf" (UID: "1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:49:08.248617 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:08.248576    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:08.249109 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:08.248714    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:08.249109 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:08.248791    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:08.249109 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:08.248873    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:08.249109 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:08.248937    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:08.249109 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:08.249015    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:09.366927 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:09.366656    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r" event={"ID":"65f83d8a-4cf8-4682-be48-d0274b9d0670","Type":"ContainerStarted","Data":"2af47c96c597ba23008d35c6e67c609b6be5df1fcac2cff05d991b8659798724"}
Jul 23 15:49:09.370283 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:09.369546    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p995g" event={"ID":"f1a0d25d-5f09-4812-9861-7678dc1af972","Type":"ContainerStarted","Data":"e5c5a24cf6f00e1ae0f76ec5cb1486847a91af3d44748b3d6e19c42d6bf3ff3c"}
Jul 23 15:49:09.372654 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:09.371933    2562 generic.go:356] "Generic (PLEG): container finished" podID="07e7e5bd-7e31-41af-8981-875038b497bc" containerID="ce956e50576128989a859fdba1e541057210335061689b8d9fa6466552119639" exitCode=0
Jul 23 15:49:09.372654 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:09.372012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97djp" event={"ID":"07e7e5bd-7e31-41af-8981-875038b497bc","Type":"ContainerDied","Data":"ce956e50576128989a859fdba1e541057210335061689b8d9fa6466552119639"}
Jul 23 15:49:09.375973 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:09.375915    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-99t5l" event={"ID":"d451b749-46af-4de3-9409-ad4978d400ee","Type":"ContainerStarted","Data":"2c33c3c02fdb31e42843506285f31764579291df0fffb8eaa733df74f76c4404"}
Jul 23 15:49:09.379140 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:09.379109    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9cc8m" event={"ID":"578de781-ab34-4bfe-b94a-fb313157b5d2","Type":"ContainerStarted","Data":"0faaf0fa436716197b215c739d406d04781dd0096939e9573435e058679155e7"}
Jul 23 15:49:09.381681 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:09.381417    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-564fg" event={"ID":"cae772ab-5fcb-444a-b8c9-79e8048348e1","Type":"ContainerStarted","Data":"928b090fcff5dca5d1dcad43ea92c510d3e2e6fbf3624952263da2590332b0de"}
Jul 23 15:49:09.417720 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:09.417574    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-p995g" podStartSLOduration=3.932588049 podStartE2EDuration="12.417550646s" totalImagesPullingTime="8.484962597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:09.393081585 +0000 UTC m=+12.597448979" watchObservedRunningTime="2026-07-23 15:49:09.417550646 +0000 UTC m=+12.621918042"
Jul 23 15:49:09.435428 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:09.435363    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-99t5l" podStartSLOduration=3.981043686 podStartE2EDuration="12.435345187s" totalImagesPullingTime="8.454301501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:09.417200068 +0000 UTC m=+12.621567472" watchObservedRunningTime="2026-07-23 15:49:09.435345187 +0000 UTC m=+12.639712648"
Jul 23 15:49:09.435690 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:09.435656    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-564fg" podStartSLOduration=3.84594163 podStartE2EDuration="12.435644486s" totalImagesPullingTime="8.589702856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:09.435270535 +0000 UTC m=+12.639637930" watchObservedRunningTime="2026-07-23 15:49:09.435644486 +0000 UTC m=+12.640011880"
Jul 23 15:49:09.457029 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:09.456469    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9cc8m" podStartSLOduration=3.9132898000000003 podStartE2EDuration="12.456448767s" totalImagesPullingTime="8.543158967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:09.456447776 +0000 UTC m=+12.660815172" watchObservedRunningTime="2026-07-23 15:49:09.456448767 +0000 UTC m=+12.660816161"
Jul 23 15:49:10.248256 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:10.248216    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:10.248256 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:10.248255    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:10.248507 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:10.248231    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:10.248507 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:10.248342    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:10.248507 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:10.248465    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:10.248661 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:10.248552    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:11.387639 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:11.387599    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jq948" event={"ID":"4df03883-9eeb-4b8b-9a6f-556bdcba6eb2","Type":"ContainerStarted","Data":"6a956f41eea35fa1e0440ca2a2d2e5e8d6e00005cf6488b8db79e3c29aca392f"}
Jul 23 15:49:11.416831 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:11.416755    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jq948" podStartSLOduration=5.923243288 podStartE2EDuration="14.416733232s" totalImagesPullingTime="8.493489944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:11.416138121 +0000 UTC m=+14.620505720" watchObservedRunningTime="2026-07-23 15:49:11.416733232 +0000 UTC m=+14.621100627"
Jul 23 15:49:12.248314 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:12.248273    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:12.248314 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:12.248317    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:12.248546 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:12.248389    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:12.248546 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:12.248412    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:12.248546 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:12.248509    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:12.248658 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:12.248609    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:12.601192 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:12.601109    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:49:12.601869 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:12.601850    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:49:13.392849 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:13.392789    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:49:13.393668 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:13.393638    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-99t5l"
Jul 23 15:49:14.248314 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:14.248282    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:14.249045 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:14.248278    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:14.249045 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:14.248422    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:14.249045 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:14.248291    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:14.249045 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:14.248493    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:14.249045 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:14.248567    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:14.831486 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:14.831448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:14.831675 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:14.831585    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 15:49:14.831675 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:14.831653    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret podName:9b20f8df-5ab7-4534-8ae3-f51228f128b8 nodeName:}" failed. No retries permitted until 2026-07-23 15:49:30.831632131 +0000 UTC m=+34.035999516 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret") pod "global-pull-secret-syncer-sl96g" (UID: "9b20f8df-5ab7-4534-8ae3-f51228f128b8") : object "kube-system"/"original-pull-secret" not registered
Jul 23 15:49:14.931929 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:14.931894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:14.932131 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:14.932053    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:49:14.932131 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:14.932124    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs podName:222a0a15-7ded-4543-94e0-21ddcd1bcf3d nodeName:}" failed. No retries permitted until 2026-07-23 15:49:30.932104246 +0000 UTC m=+34.136471621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs") pod "network-metrics-daemon-cnjpz" (UID: "222a0a15-7ded-4543-94e0-21ddcd1bcf3d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:49:15.032389 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:15.032253    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwmmj\" (UniqueName: \"kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj\") pod \"network-check-target-zhhqf\" (UID: \"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441\") " pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:15.032590 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:15.032510    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 15:49:15.032590 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:15.032547    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 15:49:15.032590 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:15.032562    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wwmmj for pod openshift-network-diagnostics/network-check-target-zhhqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:49:15.032759 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:15.032633    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj podName:1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441 nodeName:}" failed. No retries permitted until 2026-07-23 15:49:31.032612607 +0000 UTC m=+34.236979983 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwmmj" (UniqueName: "kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj") pod "network-check-target-zhhqf" (UID: "1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:49:16.248471 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:16.248427    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:16.249043 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:16.248478    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:16.249043 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:16.248556    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:16.249043 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:16.248578    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:16.249043 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:16.248638    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:16.249043 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:16.248683    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:18.247719 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:18.247684    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:18.248161 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:18.247827    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:18.248161 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:18.247838    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:18.248161 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:18.247859    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:18.248161 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:18.247940    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:18.248161 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:18.248022    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:19.407102 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:19.406828    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-57vf5" event={"ID":"ad0c93e9-d754-43b4-9310-125f0fe25a9e","Type":"ContainerStarted","Data":"945f6b0b32d42550a6fb772cc1532def6fc9cdc67dab78d2c49d2472a0945351"}
Jul 23 15:49:20.248339 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:20.248299    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:20.248725 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:20.248353    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:20.248725 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:20.248353    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:20.248725 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:20.248423    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:20.248725 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:20.248544    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:20.248725 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:20.248638    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:21.615445 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:21.615417    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 15:49:22.152569 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.152341    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T15:49:21.615438362Z","UUID":"4cf4f303-bce6-47af-b111-f1eb95eea22c","Handler":null,"Name":"","Endpoint":""}
Jul 23 15:49:22.153750 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.153733    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 15:49:22.153849 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.153768    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 15:49:22.248166 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.248077    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:22.248314 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.248185    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:22.248314 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:22.248192    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:22.248314 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.248218    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:22.248314 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:22.248296    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:22.248444 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:22.248387    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:22.414541 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.414504    2562 generic.go:356] "Generic (PLEG): container finished" podID="07e7e5bd-7e31-41af-8981-875038b497bc" containerID="6008d3afd58cb2fccc3e66cb1e73cdfe4338a4ad766c7da01226c9caa7b3efa4" exitCode=0
Jul 23 15:49:22.414714 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.414593    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97djp" event={"ID":"07e7e5bd-7e31-41af-8981-875038b497bc","Type":"ContainerDied","Data":"6008d3afd58cb2fccc3e66cb1e73cdfe4338a4ad766c7da01226c9caa7b3efa4"}
Jul 23 15:49:22.417593 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.417572    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 15:49:22.417937 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.417919    2562 generic.go:356] "Generic (PLEG): container finished" podID="d14e04e1-071c-4b0e-8a00-188ba9797d1a" containerID="5fea1a1a4a2aadc2182910ce603e80054ed4783032e350eff8f81e4dff16089c" exitCode=1
Jul 23 15:49:22.418003 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.417983    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr" event={"ID":"d14e04e1-071c-4b0e-8a00-188ba9797d1a","Type":"ContainerStarted","Data":"977c0704cc128fa0d73d397632024cedf948f31978625a50807c91a6ec9fdf58"}
Jul 23 15:49:22.418059 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.418040    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr" event={"ID":"d14e04e1-071c-4b0e-8a00-188ba9797d1a","Type":"ContainerStarted","Data":"a9f95020c66742fb3691f239376cbdfc627d010450c4f24e892369d713229233"}
Jul 23 15:49:22.418108 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.418095    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr" event={"ID":"d14e04e1-071c-4b0e-8a00-188ba9797d1a","Type":"ContainerStarted","Data":"7ee29e0cf938a56f6358516e93a83999f26f1d6fc0d5aed9e013b9dadf1468ad"}
Jul 23 15:49:22.418146 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.418139    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr" event={"ID":"d14e04e1-071c-4b0e-8a00-188ba9797d1a","Type":"ContainerStarted","Data":"e64b93fc30491735de74343b478aae1177b6bbc05ce082fa465ea6a24bc25045"}
Jul 23 15:49:22.418227 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.418184    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr" event={"ID":"d14e04e1-071c-4b0e-8a00-188ba9797d1a","Type":"ContainerDied","Data":"5fea1a1a4a2aadc2182910ce603e80054ed4783032e350eff8f81e4dff16089c"}
Jul 23 15:49:22.418227 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.418221    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr" event={"ID":"d14e04e1-071c-4b0e-8a00-188ba9797d1a","Type":"ContainerStarted","Data":"06008f5a0298d2783d7618265e631f8215dcc71f749c78c363f19df33c9445fb"}
Jul 23 15:49:22.419691 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.419672    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r" event={"ID":"65f83d8a-4cf8-4682-be48-d0274b9d0670","Type":"ContainerStarted","Data":"d52754ce0d56762841497298944d83b40aef76b7cbcd7b0af00a31bbd7eccab8"}
Jul 23 15:49:22.439061 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:22.439000    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-57vf5" podStartSLOduration=6.916404241 podStartE2EDuration="25.438985697s" totalImagesPullingTime="18.522581456s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:19.429837685 +0000 UTC m=+22.634205079" watchObservedRunningTime="2026-07-23 15:49:22.438985697 +0000 UTC m=+25.643353089"
Jul 23 15:49:24.247945 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:24.247915    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:24.248587 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:24.247952    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:24.248587 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:24.247929    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:24.248587 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:24.248043    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:24.248587 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:24.248480    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:24.248873 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:24.248743    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:24.426073 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:24.426042    2562 generic.go:356] "Generic (PLEG): container finished" podID="07e7e5bd-7e31-41af-8981-875038b497bc" containerID="1e80f9d02046dc689d60ee7233aee6fcfedd4efbd5bdaa074f05dc3c285a7b53" exitCode=0
Jul 23 15:49:24.426245 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:24.426113    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97djp" event={"ID":"07e7e5bd-7e31-41af-8981-875038b497bc","Type":"ContainerDied","Data":"1e80f9d02046dc689d60ee7233aee6fcfedd4efbd5bdaa074f05dc3c285a7b53"}
Jul 23 15:49:24.429167 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:24.429123    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 15:49:24.429476 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:24.429455    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr" event={"ID":"d14e04e1-071c-4b0e-8a00-188ba9797d1a","Type":"ContainerStarted","Data":"a01a4e101707a73f8484a807c9eca25237318537990cdfc5038e6f8aeffd14b1"}
Jul 23 15:49:25.436862 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:25.436830    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r" event={"ID":"65f83d8a-4cf8-4682-be48-d0274b9d0670","Type":"ContainerStarted","Data":"d69680b93c8f1f452ae6c89865411cf812bd759a56a38efa030e8a4a3a37ffe6"}
Jul 23 15:49:25.456121 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:25.456072    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-98n6r" podStartSLOduration=4.815246696 podStartE2EDuration="28.456054852s" totalImagesPullingTime="23.640808156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:25.455832221 +0000 UTC m=+28.660199613" watchObservedRunningTime="2026-07-23 15:49:25.456054852 +0000 UTC m=+28.660422243"
Jul 23 15:49:26.248631 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:26.248599    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:26.248853 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:26.248602    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:26.248853 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:26.248710    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:26.248853 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:26.248729    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:26.248853 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:26.248841    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:26.249037 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:26.248924    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:26.442515 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:26.442237    2562 generic.go:356] "Generic (PLEG): container finished" podID="07e7e5bd-7e31-41af-8981-875038b497bc" containerID="4b7cbe07f879021c55be2b981eaa3cd30d680ba08556b3270b89b99df204e303" exitCode=0
Jul 23 15:49:26.442515 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:26.442323    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97djp" event={"ID":"07e7e5bd-7e31-41af-8981-875038b497bc","Type":"ContainerDied","Data":"4b7cbe07f879021c55be2b981eaa3cd30d680ba08556b3270b89b99df204e303"}
Jul 23 15:49:26.447894 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:26.446109    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 15:49:26.447894 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:26.446550    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr" event={"ID":"d14e04e1-071c-4b0e-8a00-188ba9797d1a","Type":"ContainerStarted","Data":"6ffbd764d3b849cd4871a972406efb6c850ef164b3292afd94129b8118835d63"}
Jul 23 15:49:26.447894 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:26.447106    2562 scope.go:117] "RemoveContainer" containerID="5fea1a1a4a2aadc2182910ce603e80054ed4783032e350eff8f81e4dff16089c"
Jul 23 15:49:27.453331 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:27.453123    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 15:49:27.453731 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:27.453703    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr" event={"ID":"d14e04e1-071c-4b0e-8a00-188ba9797d1a","Type":"ContainerStarted","Data":"040445ca006f60164b6316bc8bdad0b48d36b6ada4d6b4656b3e2e9816f69e8f"}
Jul 23 15:49:27.454067 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:27.454045    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:49:27.454176 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:27.454120    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:49:27.454176 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:27.454170    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:49:27.472593 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:27.472517    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:49:27.472954 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:27.472932    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:49:27.487444 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:27.487398    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr" podStartSLOduration=9.455662622 podStartE2EDuration="30.487382938s" totalImagesPullingTime="21.031720316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:27.485605836 +0000 UTC m=+30.689973230" watchObservedRunningTime="2026-07-23 15:49:27.487382938 +0000 UTC m=+30.691750330"
Jul 23 15:49:28.248590 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:28.248555    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:28.248590 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:28.248582    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:28.248590 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:28.248569    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:28.248820 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:28.248678    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:28.248820 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:28.248734    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:28.248891 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:28.248796    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:30.247784 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:30.247753    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:30.248253 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:30.247814    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:30.248253 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:30.247883    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:30.248253 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:30.247902    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:30.248253 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:30.248002    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:30.248253 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:30.248074    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:30.844723 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:30.844679    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:30.844976 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:30.844864    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 15:49:30.844976 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:30.844949    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret podName:9b20f8df-5ab7-4534-8ae3-f51228f128b8 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:02.844928721 +0000 UTC m=+66.049296113 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret") pod "global-pull-secret-syncer-sl96g" (UID: "9b20f8df-5ab7-4534-8ae3-f51228f128b8") : object "kube-system"/"original-pull-secret" not registered
Jul 23 15:49:30.946065 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:30.946034    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:30.946232 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:30.946149    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:49:30.946232 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:30.946202    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs podName:222a0a15-7ded-4543-94e0-21ddcd1bcf3d nodeName:}" failed. No retries permitted until 2026-07-23 15:50:02.946187484 +0000 UTC m=+66.150554855 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs") pod "network-metrics-daemon-cnjpz" (UID: "222a0a15-7ded-4543-94e0-21ddcd1bcf3d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 15:49:31.046643 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:31.046601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwmmj\" (UniqueName: \"kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj\") pod \"network-check-target-zhhqf\" (UID: \"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441\") " pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:31.046907 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:31.046774    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 15:49:31.046907 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:31.046796    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 15:49:31.046907 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:31.046822    2562 projected.go:196] Error preparing data for projected volume kube-api-access-wwmmj for pod openshift-network-diagnostics/network-check-target-zhhqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:49:31.046907 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:31.046874    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj podName:1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:03.046861043 +0000 UTC m=+66.251228415 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwmmj" (UniqueName: "kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj") pod "network-check-target-zhhqf" (UID: "1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 15:49:31.468866 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:31.468825    2562 generic.go:356] "Generic (PLEG): container finished" podID="07e7e5bd-7e31-41af-8981-875038b497bc" containerID="c4239a2d69e3f17527e7007b458c845c2ef5765ade0032e49e4e4e1b400588e8" exitCode=0
Jul 23 15:49:31.469255 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:31.468899    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97djp" event={"ID":"07e7e5bd-7e31-41af-8981-875038b497bc","Type":"ContainerDied","Data":"c4239a2d69e3f17527e7007b458c845c2ef5765ade0032e49e4e4e1b400588e8"}
Jul 23 15:49:32.248293 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:32.248250    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:32.248293 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:32.248274    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:32.248293 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:32.248299    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:32.248544 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:32.248367    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:32.248544 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:32.248421    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:32.248544 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:32.248479    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:32.473897 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:32.473865    2562 generic.go:356] "Generic (PLEG): container finished" podID="07e7e5bd-7e31-41af-8981-875038b497bc" containerID="0cae398298dcb2ac73502b70489b8f49c57f5e7cc9928bf2c8b167ab60a051c4" exitCode=0
Jul 23 15:49:32.474255 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:32.473919    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97djp" event={"ID":"07e7e5bd-7e31-41af-8981-875038b497bc","Type":"ContainerDied","Data":"0cae398298dcb2ac73502b70489b8f49c57f5e7cc9928bf2c8b167ab60a051c4"}
Jul 23 15:49:33.344023 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:33.343823    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zhhqf"]
Jul 23 15:49:33.344239 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:33.344153    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:33.344313 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:33.344254    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:33.346110 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:33.346065    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sl96g"]
Jul 23 15:49:33.346233 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:33.346182    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:33.346301 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:33.346276    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:33.346696 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:33.346636    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cnjpz"]
Jul 23 15:49:33.346793 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:33.346740    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:33.346888 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:33.346867    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:33.479992 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:33.479955    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97djp" event={"ID":"07e7e5bd-7e31-41af-8981-875038b497bc","Type":"ContainerStarted","Data":"21e958165f11b69f669483837e61beb053bc1fa4ade2066f4e295863e6c83ab2"}
Jul 23 15:49:33.504331 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:33.504271    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-97djp" podStartSLOduration=8.872497972 podStartE2EDuration="36.504250098s" totalImagesPullingTime="27.631752126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:33.503684737 +0000 UTC m=+36.708052130" watchObservedRunningTime="2026-07-23 15:49:33.504250098 +0000 UTC m=+36.708617492"
Jul 23 15:49:35.248114 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:35.248078    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:35.248684 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:35.248078    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:35.248684 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:35.248191    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhhqf" podUID="1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441"
Jul 23 15:49:35.248684 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:35.248210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:35.248684 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:35.248286    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnjpz" podUID="222a0a15-7ded-4543-94e0-21ddcd1bcf3d"
Jul 23 15:49:35.248684 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:35.248350    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl96g" podUID="9b20f8df-5ab7-4534-8ae3-f51228f128b8"
Jul 23 15:49:36.576710 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.576680    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeReady"
Jul 23 15:49:36.577199 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.576862    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 15:49:36.636862 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.636781    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-979hc"]
Jul 23 15:49:36.662217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.662165    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gvkpg"]
Jul 23 15:49:36.662403 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.662300    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.665380 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.665354    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 15:49:36.667187 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.667163    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tmhs6\""
Jul 23 15:49:36.667319 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.667246    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 15:49:36.668528 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.668295    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 15:49:36.668528 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.668408    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 15:49:36.676077 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.676053    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-979hc"]
Jul 23 15:49:36.676178 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.676088    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gvkpg"]
Jul 23 15:49:36.676178 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.676167    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.681966 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.681945    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-94s22\""
Jul 23 15:49:36.682073 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.681946    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 15:49:36.682073 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.682043    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 15:49:36.751471 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.751438    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-695p5"]
Jul 23 15:49:36.777709 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.777665    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-695p5"
Jul 23 15:49:36.778922 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.778896    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-695p5"]
Jul 23 15:49:36.780251 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.780228    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jv7t4\""
Jul 23 15:49:36.780510 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.780482    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 15:49:36.780632 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.780556    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 15:49:36.780707 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.780660    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 15:49:36.792634 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.792608    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e31c4db0-378a-4f21-aa9e-24f4ed93743a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.792722 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.792644    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e31c4db0-378a-4f21-aa9e-24f4ed93743a-data-volume\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.792722 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.792663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0074ed98-eb79-4d56-9359-0dbf13b31760-tmp-dir\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.792722 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.792682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0074ed98-eb79-4d56-9359-0dbf13b31760-metrics-tls\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.792722 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.792708    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e31c4db0-378a-4f21-aa9e-24f4ed93743a-crio-socket\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.792880 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.792731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzfcj\" (UniqueName: \"kubernetes.io/projected/0074ed98-eb79-4d56-9359-0dbf13b31760-kube-api-access-nzfcj\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.792880 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.792822    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e31c4db0-378a-4f21-aa9e-24f4ed93743a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.792880 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.792847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tjhv\" (UniqueName: \"kubernetes.io/projected/e31c4db0-378a-4f21-aa9e-24f4ed93743a-kube-api-access-8tjhv\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.792880 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.792873    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0074ed98-eb79-4d56-9359-0dbf13b31760-config-volume\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.894249 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894161    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e31c4db0-378a-4f21-aa9e-24f4ed93743a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.894249 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e31c4db0-378a-4f21-aa9e-24f4ed93743a-data-volume\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.894249 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0074ed98-eb79-4d56-9359-0dbf13b31760-tmp-dir\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.894249 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894245    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e24b669-70a0-4855-8cbe-a476295a950a-cert\") pod \"ingress-canary-695p5\" (UID: \"6e24b669-70a0-4855-8cbe-a476295a950a\") " pod="openshift-ingress-canary/ingress-canary-695p5"
Jul 23 15:49:36.894571 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894369    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0074ed98-eb79-4d56-9359-0dbf13b31760-metrics-tls\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.894571 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e31c4db0-378a-4f21-aa9e-24f4ed93743a-crio-socket\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.894571 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894443    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzfcj\" (UniqueName: \"kubernetes.io/projected/0074ed98-eb79-4d56-9359-0dbf13b31760-kube-api-access-nzfcj\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.894571 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894477    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e31c4db0-378a-4f21-aa9e-24f4ed93743a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.894571 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tjhv\" (UniqueName: \"kubernetes.io/projected/e31c4db0-378a-4f21-aa9e-24f4ed93743a-kube-api-access-8tjhv\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.894571 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894541    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0074ed98-eb79-4d56-9359-0dbf13b31760-config-volume\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.894571 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894555    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0074ed98-eb79-4d56-9359-0dbf13b31760-tmp-dir\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.894906 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894587    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb4kz\" (UniqueName: \"kubernetes.io/projected/6e24b669-70a0-4855-8cbe-a476295a950a-kube-api-access-zb4kz\") pod \"ingress-canary-695p5\" (UID: \"6e24b669-70a0-4855-8cbe-a476295a950a\") " pod="openshift-ingress-canary/ingress-canary-695p5"
Jul 23 15:49:36.894906 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e31c4db0-378a-4f21-aa9e-24f4ed93743a-data-volume\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.894906 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894718    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e31c4db0-378a-4f21-aa9e-24f4ed93743a-crio-socket\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.895046 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.894920    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e31c4db0-378a-4f21-aa9e-24f4ed93743a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.895116 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.895098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0074ed98-eb79-4d56-9359-0dbf13b31760-config-volume\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.898417 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.898400    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0074ed98-eb79-4d56-9359-0dbf13b31760-metrics-tls\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.898472 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.898407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e31c4db0-378a-4f21-aa9e-24f4ed93743a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.904687 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.904665    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tjhv\" (UniqueName: \"kubernetes.io/projected/e31c4db0-378a-4f21-aa9e-24f4ed93743a-kube-api-access-8tjhv\") pod \"insights-runtime-extractor-979hc\" (UID: \"e31c4db0-378a-4f21-aa9e-24f4ed93743a\") " pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.909398 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.909377    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzfcj\" (UniqueName: \"kubernetes.io/projected/0074ed98-eb79-4d56-9359-0dbf13b31760-kube-api-access-nzfcj\") pod \"dns-default-gvkpg\" (UID: \"0074ed98-eb79-4d56-9359-0dbf13b31760\") " pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.973535 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.973500    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-979hc"
Jul 23 15:49:36.983587 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.983559    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:36.995302 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.995276    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb4kz\" (UniqueName: \"kubernetes.io/projected/6e24b669-70a0-4855-8cbe-a476295a950a-kube-api-access-zb4kz\") pod \"ingress-canary-695p5\" (UID: \"6e24b669-70a0-4855-8cbe-a476295a950a\") " pod="openshift-ingress-canary/ingress-canary-695p5"
Jul 23 15:49:36.995449 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.995366    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e24b669-70a0-4855-8cbe-a476295a950a-cert\") pod \"ingress-canary-695p5\" (UID: \"6e24b669-70a0-4855-8cbe-a476295a950a\") " pod="openshift-ingress-canary/ingress-canary-695p5"
Jul 23 15:49:36.997837 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:36.997817    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e24b669-70a0-4855-8cbe-a476295a950a-cert\") pod \"ingress-canary-695p5\" (UID: \"6e24b669-70a0-4855-8cbe-a476295a950a\") " pod="openshift-ingress-canary/ingress-canary-695p5"
Jul 23 15:49:37.007917 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.007891    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb4kz\" (UniqueName: \"kubernetes.io/projected/6e24b669-70a0-4855-8cbe-a476295a950a-kube-api-access-zb4kz\") pod \"ingress-canary-695p5\" (UID: \"6e24b669-70a0-4855-8cbe-a476295a950a\") " pod="openshift-ingress-canary/ingress-canary-695p5"
Jul 23 15:49:37.074742 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.074710    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"]
Jul 23 15:49:37.087129 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.087057    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-695p5"
Jul 23 15:49:37.101985 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.101951    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"]
Jul 23 15:49:37.102142 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.102084    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"
Jul 23 15:49:37.105919 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.105307    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-x474n\""
Jul 23 15:49:37.108902 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.108180    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 23 15:49:37.181856 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.181658    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-hbd4t"]
Jul 23 15:49:37.192368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.192341    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-hbd4t"
Jul 23 15:49:37.196334 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.196293    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1079e788-8486-467b-b624-d97bec73a90a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-r65r6\" (UID: \"1079e788-8486-467b-b624-d97bec73a90a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"
Jul 23 15:49:37.197496 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.197472    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 15:49:37.197766 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.197531    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mz8kr\""
Jul 23 15:49:37.198211 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.198195    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 15:49:37.204275 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.204253    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gvkpg"]
Jul 23 15:49:37.206171 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.205496    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-hbd4t"]
Jul 23 15:49:37.206948 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.206927    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-979hc"]
Jul 23 15:49:37.210281 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:37.210254    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0074ed98_eb79_4d56_9359_0dbf13b31760.slice/crio-7cbc3ae1fec84958f3436c2e6a2612bbd35113b2fb3e79c7e41ac43051626565 WatchSource:0}: Error finding container 7cbc3ae1fec84958f3436c2e6a2612bbd35113b2fb3e79c7e41ac43051626565: Status 404 returned error can't find the container with id 7cbc3ae1fec84958f3436c2e6a2612bbd35113b2fb3e79c7e41ac43051626565
Jul 23 15:49:37.210883 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:37.210860    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode31c4db0_378a_4f21_aa9e_24f4ed93743a.slice/crio-708f343940d9d447907549f9d6dae1a908ee9493303887d605262aaee13d5ebe WatchSource:0}: Error finding container 708f343940d9d447907549f9d6dae1a908ee9493303887d605262aaee13d5ebe: Status 404 returned error can't find the container with id 708f343940d9d447907549f9d6dae1a908ee9493303887d605262aaee13d5ebe
Jul 23 15:49:37.248710 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.248675    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:49:37.248884 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.248763    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:49:37.248884 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.248812    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:49:37.252533 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.252512    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 15:49:37.252814 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.252786    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 15:49:37.253006 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.252981    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 15:49:37.253006 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.253005    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pzsj9\""
Jul 23 15:49:37.253194 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.253018    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 15:49:37.258321 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:37.258298    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e24b669_70a0_4855_8cbe_a476295a950a.slice/crio-058d5a2384567f6efee23ab986e2e8d370fc3f34853e95d25510e2a9d3374b84 WatchSource:0}: Error finding container 058d5a2384567f6efee23ab986e2e8d370fc3f34853e95d25510e2a9d3374b84: Status 404 returned error can't find the container with id 058d5a2384567f6efee23ab986e2e8d370fc3f34853e95d25510e2a9d3374b84
Jul 23 15:49:37.258838 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.258786    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bn5p9\""
Jul 23 15:49:37.259656 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.259635    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-695p5"]
Jul 23 15:49:37.297720 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.297681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1079e788-8486-467b-b624-d97bec73a90a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-r65r6\" (UID: \"1079e788-8486-467b-b624-d97bec73a90a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"
Jul 23 15:49:37.297901 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.297789    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh6lk\" (UniqueName: \"kubernetes.io/projected/a1ee8604-fbac-48a8-96d7-2d8d9648cd05-kube-api-access-qh6lk\") pod \"downloads-5bc67dcd46-hbd4t\" (UID: \"a1ee8604-fbac-48a8-96d7-2d8d9648cd05\") " pod="openshift-console/downloads-5bc67dcd46-hbd4t"
Jul 23 15:49:37.297901 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:37.297857    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 23 15:49:37.297970 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:37.297923    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1079e788-8486-467b-b624-d97bec73a90a-tls-certificates podName:1079e788-8486-467b-b624-d97bec73a90a nodeName:}" failed. No retries permitted until 2026-07-23 15:49:37.797908032 +0000 UTC m=+41.002275402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/1079e788-8486-467b-b624-d97bec73a90a-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-r65r6" (UID: "1079e788-8486-467b-b624-d97bec73a90a") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 23 15:49:37.398337 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.398303    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh6lk\" (UniqueName: \"kubernetes.io/projected/a1ee8604-fbac-48a8-96d7-2d8d9648cd05-kube-api-access-qh6lk\") pod \"downloads-5bc67dcd46-hbd4t\" (UID: \"a1ee8604-fbac-48a8-96d7-2d8d9648cd05\") " pod="openshift-console/downloads-5bc67dcd46-hbd4t"
Jul 23 15:49:37.409578 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.409544    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh6lk\" (UniqueName: \"kubernetes.io/projected/a1ee8604-fbac-48a8-96d7-2d8d9648cd05-kube-api-access-qh6lk\") pod \"downloads-5bc67dcd46-hbd4t\" (UID: \"a1ee8604-fbac-48a8-96d7-2d8d9648cd05\") " pod="openshift-console/downloads-5bc67dcd46-hbd4t"
Jul 23 15:49:37.490368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.490335    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gvkpg" event={"ID":"0074ed98-eb79-4d56-9359-0dbf13b31760","Type":"ContainerStarted","Data":"7cbc3ae1fec84958f3436c2e6a2612bbd35113b2fb3e79c7e41ac43051626565"}
Jul 23 15:49:37.491420 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.491396    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-979hc" event={"ID":"e31c4db0-378a-4f21-aa9e-24f4ed93743a","Type":"ContainerStarted","Data":"708f343940d9d447907549f9d6dae1a908ee9493303887d605262aaee13d5ebe"}
Jul 23 15:49:37.492390 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.492365    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-695p5" event={"ID":"6e24b669-70a0-4855-8cbe-a476295a950a","Type":"ContainerStarted","Data":"058d5a2384567f6efee23ab986e2e8d370fc3f34853e95d25510e2a9d3374b84"}
Jul 23 15:49:37.504755 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.504731    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-hbd4t"
Jul 23 15:49:37.659171 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.657750    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-hbd4t"]
Jul 23 15:49:37.662579 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:37.662547    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ee8604_fbac_48a8_96d7_2d8d9648cd05.slice/crio-cf163b394a6a3278699885b572554327b042a2cc3d29549e95db5eee2f5f3878 WatchSource:0}: Error finding container cf163b394a6a3278699885b572554327b042a2cc3d29549e95db5eee2f5f3878: Status 404 returned error can't find the container with id cf163b394a6a3278699885b572554327b042a2cc3d29549e95db5eee2f5f3878
Jul 23 15:49:37.801450 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:37.801359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1079e788-8486-467b-b624-d97bec73a90a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-r65r6\" (UID: \"1079e788-8486-467b-b624-d97bec73a90a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"
Jul 23 15:49:37.801610 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:37.801533    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 23 15:49:37.801686 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:37.801617    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1079e788-8486-467b-b624-d97bec73a90a-tls-certificates podName:1079e788-8486-467b-b624-d97bec73a90a nodeName:}" failed. No retries permitted until 2026-07-23 15:49:38.801594758 +0000 UTC m=+42.005962131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/1079e788-8486-467b-b624-d97bec73a90a-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-r65r6" (UID: "1079e788-8486-467b-b624-d97bec73a90a") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 23 15:49:38.497430 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:38.497388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-hbd4t" event={"ID":"a1ee8604-fbac-48a8-96d7-2d8d9648cd05","Type":"ContainerStarted","Data":"cf163b394a6a3278699885b572554327b042a2cc3d29549e95db5eee2f5f3878"}
Jul 23 15:49:38.811253 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:38.811168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1079e788-8486-467b-b624-d97bec73a90a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-r65r6\" (UID: \"1079e788-8486-467b-b624-d97bec73a90a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"
Jul 23 15:49:38.815280 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:38.815250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1079e788-8486-467b-b624-d97bec73a90a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-r65r6\" (UID: \"1079e788-8486-467b-b624-d97bec73a90a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"
Jul 23 15:49:38.919527 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:38.918347    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"
Jul 23 15:49:39.601095 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:39.601058    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"]
Jul 23 15:49:39.739607 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:39.738384    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1079e788_8486_467b_b624_d97bec73a90a.slice/crio-aefae4fe7715e4f6f609b8a5c6b692599be18556e3e3de3c267b64c34d5c32ba WatchSource:0}: Error finding container aefae4fe7715e4f6f609b8a5c6b692599be18556e3e3de3c267b64c34d5c32ba: Status 404 returned error can't find the container with id aefae4fe7715e4f6f609b8a5c6b692599be18556e3e3de3c267b64c34d5c32ba
Jul 23 15:49:40.508913 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:40.508640    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6" event={"ID":"1079e788-8486-467b-b624-d97bec73a90a","Type":"ContainerStarted","Data":"aefae4fe7715e4f6f609b8a5c6b692599be18556e3e3de3c267b64c34d5c32ba"}
Jul 23 15:49:40.511192 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:40.511160    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gvkpg" event={"ID":"0074ed98-eb79-4d56-9359-0dbf13b31760","Type":"ContainerStarted","Data":"f97de4b907b0181f5b8ed5cb8747ab7d307ded5164beeb04a102937ea36aca73"}
Jul 23 15:49:40.511334 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:40.511219    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gvkpg" event={"ID":"0074ed98-eb79-4d56-9359-0dbf13b31760","Type":"ContainerStarted","Data":"d3c5b4a3d68a047a476e75541289afd4b7bbbe07c4ca49497fb12428c25f8ea9"}
Jul 23 15:49:40.511522 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:40.511502    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:40.534371 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:40.534139    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gvkpg" podStartSLOduration=1.967084724 podStartE2EDuration="4.534113295s" totalImagesPullingTime="2.567028571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:40.533183427 +0000 UTC m=+43.737550819" watchObservedRunningTime="2026-07-23 15:49:40.534113295 +0000 UTC m=+43.738480689"
Jul 23 15:49:45.529413 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:45.529371    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-695p5" event={"ID":"6e24b669-70a0-4855-8cbe-a476295a950a","Type":"ContainerStarted","Data":"e4db109f8a7e95256bd7325549ea757321961aa38a05a267f1edd6b2af350ba3"}
Jul 23 15:49:45.534038 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:45.534007    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-979hc" event={"ID":"e31c4db0-378a-4f21-aa9e-24f4ed93743a","Type":"ContainerStarted","Data":"c37913e689040ff9d04465e3f9234e002cf89eb7687e0cd4979f63424b67759a"}
Jul 23 15:49:45.536797 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:45.536764    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6" event={"ID":"1079e788-8486-467b-b624-d97bec73a90a","Type":"ContainerStarted","Data":"47e8d398d9d72ca7c2495ee6c27f8f407842763dbdc9bcd1ee1a5fab18f672fe"}
Jul 23 15:49:45.537316 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:45.537293    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"
Jul 23 15:49:45.544257 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:45.544234    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6"
Jul 23 15:49:45.546520 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:45.546476    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-695p5" podStartSLOduration=2.279361134 podStartE2EDuration="9.546458048s" totalImagesPullingTime="7.267096914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:45.545409023 +0000 UTC m=+48.749776428" watchObservedRunningTime="2026-07-23 15:49:45.546458048 +0000 UTC m=+48.750825439"
Jul 23 15:49:45.573216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:45.572492    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-r65r6" podStartSLOduration=3.450805714 podStartE2EDuration="8.57246373s" totalImagesPullingTime="5.121658016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:49:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:45.57157161 +0000 UTC m=+48.775939003" watchObservedRunningTime="2026-07-23 15:49:45.57246373 +0000 UTC m=+48.776831118"
Jul 23 15:49:46.113528 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.112286    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"]
Jul 23 15:49:46.115640 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.115587    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.120278 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.119942    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 23 15:49:46.120278 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.120029    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 15:49:46.120278 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.120073    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 15:49:46.120278 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.120117    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 23 15:49:46.120278 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.120175    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wdbdc\""
Jul 23 15:49:46.120278 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.120218    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 15:49:46.124483 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.124251    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"]
Jul 23 15:49:46.184213 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.184176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csz4s\" (UniqueName: \"kubernetes.io/projected/b84cd16a-1a1b-4672-8269-95064802a81e-kube-api-access-csz4s\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.184399 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.184308    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b84cd16a-1a1b-4672-8269-95064802a81e-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.184399 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.184349    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b84cd16a-1a1b-4672-8269-95064802a81e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.184399 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.184383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b84cd16a-1a1b-4672-8269-95064802a81e-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.285548 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.285047    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b84cd16a-1a1b-4672-8269-95064802a81e-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.285548 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.285098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b84cd16a-1a1b-4672-8269-95064802a81e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.285548 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.285130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b84cd16a-1a1b-4672-8269-95064802a81e-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.285548 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:46.285258    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 23 15:49:46.285548 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:49:46.285318    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b84cd16a-1a1b-4672-8269-95064802a81e-prometheus-operator-tls podName:b84cd16a-1a1b-4672-8269-95064802a81e nodeName:}" failed. No retries permitted until 2026-07-23 15:49:46.785299516 +0000 UTC m=+49.989666908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b84cd16a-1a1b-4672-8269-95064802a81e-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-qp7gz" (UID: "b84cd16a-1a1b-4672-8269-95064802a81e") : secret "prometheus-operator-tls" not found
Jul 23 15:49:46.285548 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.285337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csz4s\" (UniqueName: \"kubernetes.io/projected/b84cd16a-1a1b-4672-8269-95064802a81e-kube-api-access-csz4s\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.286140 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.285904    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b84cd16a-1a1b-4672-8269-95064802a81e-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.289835 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.289792    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b84cd16a-1a1b-4672-8269-95064802a81e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.294909 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.294756    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csz4s\" (UniqueName: \"kubernetes.io/projected/b84cd16a-1a1b-4672-8269-95064802a81e-kube-api-access-csz4s\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.789641 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.789602    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b84cd16a-1a1b-4672-8269-95064802a81e-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:46.792492 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:46.792460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b84cd16a-1a1b-4672-8269-95064802a81e-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-qp7gz\" (UID: \"b84cd16a-1a1b-4672-8269-95064802a81e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:47.029026 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.028990    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"
Jul 23 15:49:47.068413 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.068339    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-796cffdbf5-gj9g5"]
Jul 23 15:49:47.071651 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.071630    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.076452 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.076064    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 15:49:47.076452 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.076076    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wgdh7\""
Jul 23 15:49:47.076452 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.076120    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 15:49:47.076452 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.076143    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 15:49:47.076452 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.076065    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 15:49:47.076452 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.076257    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 15:49:47.083564 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.083542    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796cffdbf5-gj9g5"]
Jul 23 15:49:47.092238 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.092214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwsbn\" (UniqueName: \"kubernetes.io/projected/a7df6af1-0ff1-4586-b3a3-e432b389f372-kube-api-access-pwsbn\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.092364 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.092346    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-config\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.092434 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.092384    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-serving-cert\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.092434 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.092406    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-oauth-config\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.092544 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.092442    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-service-ca\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.092544 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.092497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-oauth-serving-cert\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.193066 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.193016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwsbn\" (UniqueName: \"kubernetes.io/projected/a7df6af1-0ff1-4586-b3a3-e432b389f372-kube-api-access-pwsbn\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.193270 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.193128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-config\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.193270 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.193162    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-serving-cert\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.193270 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.193184    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-oauth-config\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.193270 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.193204    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-service-ca\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.193270 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.193224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-oauth-serving-cert\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.194072 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.194003    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-oauth-serving-cert\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.194392 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.194369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-config\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.195145 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.195121    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-service-ca\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.196431 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.196407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-serving-cert\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.196655 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.196632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-oauth-config\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.202319 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.202294    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwsbn\" (UniqueName: \"kubernetes.io/projected/a7df6af1-0ff1-4586-b3a3-e432b389f372-kube-api-access-pwsbn\") pod \"console-796cffdbf5-gj9g5\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") " pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:47.384409 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:47.384273    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:49:50.517383 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:50.517354    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gvkpg"
Jul 23 15:49:55.652509 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.652466    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cf55466c-fjr89"]
Jul 23 15:49:55.682856 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.682820    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cf55466c-fjr89"]
Jul 23 15:49:55.683014 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.682916    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.691703 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.691678    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 23 15:49:55.768428 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.768395    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbf4b\" (UniqueName: \"kubernetes.io/projected/22b77565-5841-4ebf-8c58-45330e3347fc-kube-api-access-nbf4b\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.768641 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.768445    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-serving-cert\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.768641 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.768542    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-oauth-config\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.768641 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.768565    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-console-config\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.768641 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.768605    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-trusted-ca-bundle\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.768892 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.768716    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-service-ca\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.768892 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.768752    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-oauth-serving-cert\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.869606 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.869570    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbf4b\" (UniqueName: \"kubernetes.io/projected/22b77565-5841-4ebf-8c58-45330e3347fc-kube-api-access-nbf4b\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.869827 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.869616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-serving-cert\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.869827 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.869675    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-oauth-config\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.869827 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.869695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-console-config\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.869827 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.869737    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-trusted-ca-bundle\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.869827 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.869785    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-service-ca\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.869827 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.869827    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-oauth-serving-cert\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.870570 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.870542    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-oauth-serving-cert\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.870984 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.870962    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-trusted-ca-bundle\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.871069 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.871013    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-service-ca\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.871229 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.871205    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-console-config\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.872384 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.872365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-oauth-config\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.872689 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.872668    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-serving-cert\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.879011 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.878987    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbf4b\" (UniqueName: \"kubernetes.io/projected/22b77565-5841-4ebf-8c58-45330e3347fc-kube-api-access-nbf4b\") pod \"console-7cf55466c-fjr89\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") " pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:55.995036 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:55.994999    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:49:56.572682 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:56.572650    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz"]
Jul 23 15:49:56.576389 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:56.576357    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb84cd16a_1a1b_4672_8269_95064802a81e.slice/crio-45c60a1deeafef4a46bc16bb1aa526f2609523fe3a1f4e7bb926038a8ec4f070 WatchSource:0}: Error finding container 45c60a1deeafef4a46bc16bb1aa526f2609523fe3a1f4e7bb926038a8ec4f070: Status 404 returned error can't find the container with id 45c60a1deeafef4a46bc16bb1aa526f2609523fe3a1f4e7bb926038a8ec4f070
Jul 23 15:49:56.599935 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:56.599729    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796cffdbf5-gj9g5"]
Jul 23 15:49:56.601475 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:56.601448    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7df6af1_0ff1_4586_b3a3_e432b389f372.slice/crio-b3f1363a64d229041b0f68135919bd68720a791ff6f9343a5972cb8ebf613d5f WatchSource:0}: Error finding container b3f1363a64d229041b0f68135919bd68720a791ff6f9343a5972cb8ebf613d5f: Status 404 returned error can't find the container with id b3f1363a64d229041b0f68135919bd68720a791ff6f9343a5972cb8ebf613d5f
Jul 23 15:49:56.639492 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:56.639464    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cf55466c-fjr89"]
Jul 23 15:49:56.642269 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:49:56.642247    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22b77565_5841_4ebf_8c58_45330e3347fc.slice/crio-db0a20e0c26912b10d7a8f5f4fff44fb41732cf91042406f5d7a92555ee6b755 WatchSource:0}: Error finding container db0a20e0c26912b10d7a8f5f4fff44fb41732cf91042406f5d7a92555ee6b755: Status 404 returned error can't find the container with id db0a20e0c26912b10d7a8f5f4fff44fb41732cf91042406f5d7a92555ee6b755
Jul 23 15:49:57.590424 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:57.590339    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796cffdbf5-gj9g5" event={"ID":"a7df6af1-0ff1-4586-b3a3-e432b389f372","Type":"ContainerStarted","Data":"b3f1363a64d229041b0f68135919bd68720a791ff6f9343a5972cb8ebf613d5f"}
Jul 23 15:49:57.592591 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:57.592520    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz" event={"ID":"b84cd16a-1a1b-4672-8269-95064802a81e","Type":"ContainerStarted","Data":"45c60a1deeafef4a46bc16bb1aa526f2609523fe3a1f4e7bb926038a8ec4f070"}
Jul 23 15:49:57.595393 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:57.595357    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-hbd4t" event={"ID":"a1ee8604-fbac-48a8-96d7-2d8d9648cd05","Type":"ContainerStarted","Data":"54ba75e56cf58d884a514c32b0f811cb3dcfca2f8c5cecbdb6976e14615686a8"}
Jul 23 15:49:57.596845 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:57.596354    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-hbd4t"
Jul 23 15:49:57.600574 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:57.600548    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf55466c-fjr89" event={"ID":"22b77565-5841-4ebf-8c58-45330e3347fc","Type":"ContainerStarted","Data":"db0a20e0c26912b10d7a8f5f4fff44fb41732cf91042406f5d7a92555ee6b755"}
Jul 23 15:49:57.606028 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:57.605966    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-979hc" event={"ID":"e31c4db0-378a-4f21-aa9e-24f4ed93743a","Type":"ContainerStarted","Data":"12364d2c61db9455f3d508cf5f303547bb90997e57dfd6cfee8809a173d44d8d"}
Jul 23 15:49:57.613000 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:57.612976    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-hbd4t"
Jul 23 15:49:57.614825 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:57.614753    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-hbd4t" podStartSLOduration=1.720579255 podStartE2EDuration="20.614734825s" totalImagesPullingTime="18.89415557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:49:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:49:57.613301631 +0000 UTC m=+60.817669025" watchObservedRunningTime="2026-07-23 15:49:57.614734825 +0000 UTC m=+60.819102223"
Jul 23 15:49:59.498865 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:49:59.497472    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9x5jr"
Jul 23 15:50:02.937486 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:02.937420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:50:02.941791 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:02.941761    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 15:50:02.951717 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:02.951609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9b20f8df-5ab7-4534-8ae3-f51228f128b8-original-pull-secret\") pod \"global-pull-secret-syncer-sl96g\" (UID: \"9b20f8df-5ab7-4534-8ae3-f51228f128b8\") " pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:50:03.038244 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.038207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:50:03.041590 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.041563    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 15:50:03.052056 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.052025    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222a0a15-7ded-4543-94e0-21ddcd1bcf3d-metrics-certs\") pod \"network-metrics-daemon-cnjpz\" (UID: \"222a0a15-7ded-4543-94e0-21ddcd1bcf3d\") " pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:50:03.060285 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.060242    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl96g"
Jul 23 15:50:03.074684 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.074648    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pzsj9\""
Jul 23 15:50:03.081484 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.081398    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnjpz"
Jul 23 15:50:03.139357 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.139313    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwmmj\" (UniqueName: \"kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj\") pod \"network-check-target-zhhqf\" (UID: \"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441\") " pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:50:03.143596 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.143572    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 15:50:03.153600 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.153369    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 15:50:03.164959 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.164898    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwmmj\" (UniqueName: \"kubernetes.io/projected/1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441-kube-api-access-wwmmj\") pod \"network-check-target-zhhqf\" (UID: \"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441\") " pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:50:03.369933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.369902    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bn5p9\""
Jul 23 15:50:03.377583 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:03.377547    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:50:04.369003 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.366249    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cnjpz"]
Jul 23 15:50:04.419948 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.419919    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sl96g"]
Jul 23 15:50:04.423927 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:50:04.423864    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b20f8df_5ab7_4534_8ae3_f51228f128b8.slice/crio-4c8e3d87dc8501a8f4e0d11374f295695d09990ee156308ab39a5187e856a020 WatchSource:0}: Error finding container 4c8e3d87dc8501a8f4e0d11374f295695d09990ee156308ab39a5187e856a020: Status 404 returned error can't find the container with id 4c8e3d87dc8501a8f4e0d11374f295695d09990ee156308ab39a5187e856a020
Jul 23 15:50:04.449010 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.448333    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zhhqf"]
Jul 23 15:50:04.641277 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.641198    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf55466c-fjr89" event={"ID":"22b77565-5841-4ebf-8c58-45330e3347fc","Type":"ContainerStarted","Data":"80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed"}
Jul 23 15:50:04.643768 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.643735    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-979hc" event={"ID":"e31c4db0-378a-4f21-aa9e-24f4ed93743a","Type":"ContainerStarted","Data":"3d8f37a473ba328f03cbf7d6196b42ae35676d66fc02abe5b42ea20d092552c9"}
Jul 23 15:50:04.645284 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.645252    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cnjpz" event={"ID":"222a0a15-7ded-4543-94e0-21ddcd1bcf3d","Type":"ContainerStarted","Data":"8b0acb392bc92fe4b0208a1494211bc3a9748e7e5d583dbe9bc951e324201bf7"}
Jul 23 15:50:04.647223 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.647156    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796cffdbf5-gj9g5" event={"ID":"a7df6af1-0ff1-4586-b3a3-e432b389f372","Type":"ContainerStarted","Data":"230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4"}
Jul 23 15:50:04.649493 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.649468    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz" event={"ID":"b84cd16a-1a1b-4672-8269-95064802a81e","Type":"ContainerStarted","Data":"a61d7ab1c7f4ac8d95f7bf73b405f6603f5940bc58634f83ffaa1d02f54c5608"}
Jul 23 15:50:04.649607 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.649510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz" event={"ID":"b84cd16a-1a1b-4672-8269-95064802a81e","Type":"ContainerStarted","Data":"e108ee9efef4fb965868277d5819648514fbc01e26f080bb6b53a02e353539a0"}
Jul 23 15:50:04.650822 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.650780    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zhhqf" event={"ID":"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441","Type":"ContainerStarted","Data":"3903a096ad7f4ffcb0251fc7d37efeb8d7fdb1eaec3c98424d8f03fc8d3558d4"}
Jul 23 15:50:04.652071 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.652049    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sl96g" event={"ID":"9b20f8df-5ab7-4534-8ae3-f51228f128b8","Type":"ContainerStarted","Data":"4c8e3d87dc8501a8f4e0d11374f295695d09990ee156308ab39a5187e856a020"}
Jul 23 15:50:04.663148 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.663101    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cf55466c-fjr89" podStartSLOduration=2.120957181 podStartE2EDuration="9.663082844s" totalImagesPullingTime="7.542125663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:49:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:04.661471863 +0000 UTC m=+67.865839257" watchObservedRunningTime="2026-07-23 15:50:04.663082844 +0000 UTC m=+67.867450239"
Jul 23 15:50:04.678656 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.678592    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-796cffdbf5-gj9g5" podStartSLOduration=10.087548835 podStartE2EDuration="17.678572953s" totalImagesPullingTime="7.591024118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:49:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:04.677826495 +0000 UTC m=+67.882193889" watchObservedRunningTime="2026-07-23 15:50:04.678572953 +0000 UTC m=+67.882940337"
Jul 23 15:50:04.696887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.696821    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-979hc" podStartSLOduration=1.9241964679999999 podStartE2EDuration="28.696780414s" totalImagesPullingTime="26.772583946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:49:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:04.695423938 +0000 UTC m=+67.899791343" watchObservedRunningTime="2026-07-23 15:50:04.696780414 +0000 UTC m=+67.901147809"
Jul 23 15:50:04.714027 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:04.713971    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qp7gz" podStartSLOduration=11.131000442 podStartE2EDuration="18.713950408s" totalImagesPullingTime="7.582949966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:04.712795839 +0000 UTC m=+67.917163232" watchObservedRunningTime="2026-07-23 15:50:04.713950408 +0000 UTC m=+67.918317802"
Jul 23 15:50:05.996026 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:05.995992    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:50:05.996530 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:05.996343    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:50:06.003916 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.003851    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:50:06.454432 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.454397    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"]
Jul 23 15:50:06.484093 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.484024    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"]
Jul 23 15:50:06.484277 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.484203    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.488367 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.487322    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-6b857"]
Jul 23 15:50:06.488367 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.487435    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 23 15:50:06.488367 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.487839    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dtt8c\""
Jul 23 15:50:06.488367 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.488076    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 23 15:50:06.505216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.504541    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sc4r8"]
Jul 23 15:50:06.505216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.504827    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.515067 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.514576    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 23 15:50:06.515067 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.514935    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 23 15:50:06.517471 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.517061    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 23 15:50:06.520617 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.520592    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8vkrv\""
Jul 23 15:50:06.530551 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.530527    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-6b857"]
Jul 23 15:50:06.530680 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.530662    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.533201 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.533179    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9nsfz\""
Jul 23 15:50:06.533532 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.533513    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 15:50:06.534056 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.533694    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 15:50:06.534056 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.533920    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 15:50:06.571558 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.571241    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq6vx\" (UniqueName: \"kubernetes.io/projected/e46ca500-8035-4e99-a59e-60270d18e78a-kube-api-access-kq6vx\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.571558 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.571284    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.571558 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.571323    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxr68\" (UniqueName: \"kubernetes.io/projected/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-kube-api-access-dxr68\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.571558 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.571360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.571558 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.571394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e46ca500-8035-4e99-a59e-60270d18e78a-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.571558 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.571428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.571558 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.571462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.571558 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.571492    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e46ca500-8035-4e99-a59e-60270d18e78a-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.571558 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.571524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.572152 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.571617    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.672178 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672146    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:50:06.672341 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672320    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmw6l\" (UniqueName: \"kubernetes.io/projected/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-kube-api-access-wmw6l\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.672403 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672362    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.672469 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e46ca500-8035-4e99-a59e-60270d18e78a-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.672535 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672476    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.672535 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672524    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.672638 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-wtmp\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.672638 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:50:06.672559    2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 23 15:50:06.672638 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672584    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.672638 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672614    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-tls\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.672638 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:50:06.672631    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-tls podName:e46ca500-8035-4e99-a59e-60270d18e78a nodeName:}" failed. No retries permitted until 2026-07-23 15:50:07.172609639 +0000 UTC m=+70.376977012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-6b857" (UID: "e46ca500-8035-4e99-a59e-60270d18e78a") : secret "kube-state-metrics-tls" not found
Jul 23 15:50:06.672933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672911    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e46ca500-8035-4e99-a59e-60270d18e78a-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.673010 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672960    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.673010 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.672995    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-textfile\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.673106 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.673034    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.673106 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.673099    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.673205 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.673130    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-root\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.673310 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.673292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-metrics-client-ca\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.673369 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.673351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq6vx\" (UniqueName: \"kubernetes.io/projected/e46ca500-8035-4e99-a59e-60270d18e78a-kube-api-access-kq6vx\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.673369 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.673359    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e46ca500-8035-4e99-a59e-60270d18e78a-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.673483 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.673387    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.673483 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.673415    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxr68\" (UniqueName: \"kubernetes.io/projected/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-kube-api-access-dxr68\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.673483 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.673442    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-sys\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.673683 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.673627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.673842 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.673824    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e46ca500-8035-4e99-a59e-60270d18e78a-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.674354 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.674309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.676354 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.676308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.676452 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.676380    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.679263 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.679226    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.701205 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.701172    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxr68\" (UniqueName: \"kubernetes.io/projected/ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e-kube-api-access-dxr68\") pod \"openshift-state-metrics-79f6958cdf-gq2vk\" (UID: \"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.705850 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.705738    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq6vx\" (UniqueName: \"kubernetes.io/projected/e46ca500-8035-4e99-a59e-60270d18e78a-kube-api-access-kq6vx\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:06.751863 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.751133    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-796cffdbf5-gj9g5"]
Jul 23 15:50:06.775152 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.774042    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.775152 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.774144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-wtmp\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.775152 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.774183    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-tls\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.775152 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.774240    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-textfile\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.775152 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.774275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.775152 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.774303    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-root\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.775579 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.775493    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-wtmp\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.776502 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.776122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-metrics-client-ca\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.776502 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.776201    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-sys\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.776502 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.776229    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmw6l\" (UniqueName: \"kubernetes.io/projected/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-kube-api-access-wmw6l\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.776868 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.776574    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.776868 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:50:06.776761    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 23 15:50:06.776868 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:50:06.776846    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-tls podName:35f95106-7910-4fd8-9f81-2f9c2ac2d0f7 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:07.276827803 +0000 UTC m=+70.481195174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-tls") pod "node-exporter-sc4r8" (UID: "35f95106-7910-4fd8-9f81-2f9c2ac2d0f7") : secret "node-exporter-tls" not found
Jul 23 15:50:06.778161 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.776968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-root\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.778161 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.777322    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-textfile\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.778161 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.777885    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-metrics-client-ca\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.778161 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.777890    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-sys\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.779947 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.779920    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.790621 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.790589    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmw6l\" (UniqueName: \"kubernetes.io/projected/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-kube-api-access-wmw6l\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:06.798473 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.798445    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"
Jul 23 15:50:06.986659 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:06.986635    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk"]
Jul 23 15:50:07.180699 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.180665    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:07.185259 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.185233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e46ca500-8035-4e99-a59e-60270d18e78a-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-6b857\" (UID: \"e46ca500-8035-4e99-a59e-60270d18e78a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:07.270791 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:50:07.270713    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad4e55e7_b5c3_4d0b_9eee_ccec23b7f30e.slice/crio-7cb13902add5f027f7ebe8425a56cb97d033cbc3974bd0a5b89c7e825bee0801 WatchSource:0}: Error finding container 7cb13902add5f027f7ebe8425a56cb97d033cbc3974bd0a5b89c7e825bee0801: Status 404 returned error can't find the container with id 7cb13902add5f027f7ebe8425a56cb97d033cbc3974bd0a5b89c7e825bee0801
Jul 23 15:50:07.281981 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.281956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-tls\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:07.284364 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.284336    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35f95106-7910-4fd8-9f81-2f9c2ac2d0f7-node-exporter-tls\") pod \"node-exporter-sc4r8\" (UID: \"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7\") " pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:07.385033 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.385008    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:50:07.426006 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.425985    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857"
Jul 23 15:50:07.443316 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.443291    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sc4r8"
Jul 23 15:50:07.573432 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.573400    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 15:50:07.589668 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.589637    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.594852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.593471    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lz4sl\""
Jul 23 15:50:07.594852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.593471    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 23 15:50:07.594852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.594037    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 23 15:50:07.594852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.594391    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 23 15:50:07.594852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.594588    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 23 15:50:07.594852 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.594703    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 23 15:50:07.595216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.594999    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 23 15:50:07.596422 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.595585    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-6b857"]
Jul 23 15:50:07.596422 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.595677    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 23 15:50:07.596422 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.595900    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 23 15:50:07.596422 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.596288    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 23 15:50:07.599819 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.599777    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 15:50:07.669872 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.669831    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk" event={"ID":"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e","Type":"ContainerStarted","Data":"102c803727a08322b6425958eec73dcc762bfce264aa8d5f307dd2bf213cb869"}
Jul 23 15:50:07.670028 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.669890    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk" event={"ID":"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e","Type":"ContainerStarted","Data":"fbf1f56660c7b3c7ce63698a722d27018ce67a2b23a29a699c3b89663a362604"}
Jul 23 15:50:07.670028 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.669910    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk" event={"ID":"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e","Type":"ContainerStarted","Data":"7cb13902add5f027f7ebe8425a56cb97d033cbc3974bd0a5b89c7e825bee0801"}
Jul 23 15:50:07.684423 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684388    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-out\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.684597 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684433    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.684597 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684455    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.684597 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684527    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-web-config\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.684597 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684581    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.684826 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684618    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.684826 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.684826 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684713    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rnr9\" (UniqueName: \"kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-kube-api-access-4rnr9\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.684826 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684763    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.684826 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684789    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.685076 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.685076 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684893    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.685076 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.684937    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.785703 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.785616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-web-config\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.785915 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.785770    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.785988 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.785944    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.786044 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.786012    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.786044 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.786038    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rnr9\" (UniqueName: \"kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-kube-api-access-4rnr9\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.786151 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.786100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.788515 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.786636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.788515 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.786706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.788515 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.786767    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.788515 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.786874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.788515 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.786927    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-out\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.788515 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.786997    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.788515 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.787044    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.788515 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.788432    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.789053 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.788605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.789053 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.788792    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.789053 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.788950    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.789398 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.789349    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-web-config\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.790001 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.789958    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.791016 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.790994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.791702 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.791657    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-out\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.791892 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.791875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.791990 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.791972    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.792029 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.791999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.792329 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.792311    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.796720 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.796701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rnr9\" (UniqueName: \"kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-kube-api-access-4rnr9\") pod \"alertmanager-main-0\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:07.905964 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:07.905931    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:50:08.608376 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.608340    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6d8984d499-4m7rc"]
Jul 23 15:50:08.632577 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.632550    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d8984d499-4m7rc"]
Jul 23 15:50:08.632745 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.632650    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.635816 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.635707    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 23 15:50:08.635816 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.635750    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 23 15:50:08.636109 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.636072    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 23 15:50:08.636204 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.636134    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bsjfl\""
Jul 23 15:50:08.636304 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.636285    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fpfv2i823rool\""
Jul 23 15:50:08.636478 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.636334    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 23 15:50:08.636478 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.636336    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 23 15:50:08.794333 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.794290    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-metrics-client-ca\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.794607 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.794353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwrts\" (UniqueName: \"kubernetes.io/projected/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-kube-api-access-xwrts\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.794607 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.794528    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-grpc-tls\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.794607 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.794575    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.794782 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.794626    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-tls\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.794782 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.794681    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.794782 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.794747    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.794966 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.794877    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.895863 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.895757    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwrts\" (UniqueName: \"kubernetes.io/projected/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-kube-api-access-xwrts\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.896033 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.895868    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-grpc-tls\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.896033 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.895912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.896033 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.895940    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-tls\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.896033 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.895966    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.896033 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.896003    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.896310 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.896039    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.896310 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.896073    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-metrics-client-ca\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.896832 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.896784    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-metrics-client-ca\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.898965 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.898941    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.899104 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.898946    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-tls\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.899163 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.899108    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.899273 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.899247    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.900097 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.900071    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.900271 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.900249    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-secret-grpc-tls\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.904245 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.904223    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwrts\" (UniqueName: \"kubernetes.io/projected/5c1bc9a7-99f1-4dba-b6b1-cc797948f674-kube-api-access-xwrts\") pod \"thanos-querier-6d8984d499-4m7rc\" (UID: \"5c1bc9a7-99f1-4dba-b6b1-cc797948f674\") " pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:08.943846 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:08.943813    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:09.133755 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:50:09.133722    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode46ca500_8035_4e99_a59e_60270d18e78a.slice/crio-c187c4691633a1fe98b1823f7c6c2e08e9caba1989e4c12328f4a56c0830c9c4 WatchSource:0}: Error finding container c187c4691633a1fe98b1823f7c6c2e08e9caba1989e4c12328f4a56c0830c9c4: Status 404 returned error can't find the container with id c187c4691633a1fe98b1823f7c6c2e08e9caba1989e4c12328f4a56c0830c9c4
Jul 23 15:50:09.679880 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:09.679835    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857" event={"ID":"e46ca500-8035-4e99-a59e-60270d18e78a","Type":"ContainerStarted","Data":"c187c4691633a1fe98b1823f7c6c2e08e9caba1989e4c12328f4a56c0830c9c4"}
Jul 23 15:50:10.861123 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:10.861091    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-76785b994f-zdt2n"]
Jul 23 15:50:10.885266 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:10.885238    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76785b994f-zdt2n"]
Jul 23 15:50:10.885417 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:10.885354    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:10.890006 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:10.889720    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 23 15:50:10.890006 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:10.889769    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 23 15:50:10.890006 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:10.889816    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 23 15:50:10.890272 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:10.890158    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-rhx7r\""
Jul 23 15:50:10.890272 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:10.890196    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1407gva1hhve0\""
Jul 23 15:50:10.890378 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:10.890163    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 23 15:50:10.998249 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:50:10.998175    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35f95106_7910_4fd8_9f81_2f9c2ac2d0f7.slice/crio-e558830b60aac1a5818ec4004bfe6654c07640a408b3b1ad9d3911114b8db3dc WatchSource:0}: Error finding container e558830b60aac1a5818ec4004bfe6654c07640a408b3b1ad9d3911114b8db3dc: Status 404 returned error can't find the container with id e558830b60aac1a5818ec4004bfe6654c07640a408b3b1ad9d3911114b8db3dc
Jul 23 15:50:11.014397 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.014362    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.014553 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.014405    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-secret-metrics-server-tls\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.014553 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.014454    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-secret-metrics-server-client-certs\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.014553 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.014546    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gz4g\" (UniqueName: \"kubernetes.io/projected/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-kube-api-access-8gz4g\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.014723 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.014579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-metrics-server-audit-profiles\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.014723 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.014619    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-audit-log\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.014723 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.014642    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-client-ca-bundle\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.115588 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.115109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gz4g\" (UniqueName: \"kubernetes.io/projected/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-kube-api-access-8gz4g\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.115588 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.115157    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-metrics-server-audit-profiles\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.115588 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.115195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-audit-log\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.120125 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.116952    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-client-ca-bundle\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.120125 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.117000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.120125 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.117031    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-secret-metrics-server-tls\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.120125 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.117100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-secret-metrics-server-client-certs\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.120125 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.117125    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-metrics-server-audit-profiles\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.120125 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.115690    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-audit-log\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.120125 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.118600    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.128552 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.128462    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-secret-metrics-server-client-certs\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.129561 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.129501    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-client-ca-bundle\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.129680 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.129593    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-secret-metrics-server-tls\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.130941 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.130874    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gz4g\" (UniqueName: \"kubernetes.io/projected/e735cb6a-9f17-4603-9ac3-06f64f8d88c6-kube-api-access-8gz4g\") pod \"metrics-server-76785b994f-zdt2n\" (UID: \"e735cb6a-9f17-4603-9ac3-06f64f8d88c6\") " pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.197243 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.197204    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:11.203598 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.203502    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 15:50:11.209345 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:50:11.209279    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa2732c_ace1_4462_abde_fb0ad81b0d4d.slice/crio-c365e3db3aa8fc813941e68ca097664cfebdd5a27e29760932d521e4da75b8a1 WatchSource:0}: Error finding container c365e3db3aa8fc813941e68ca097664cfebdd5a27e29760932d521e4da75b8a1: Status 404 returned error can't find the container with id c365e3db3aa8fc813941e68ca097664cfebdd5a27e29760932d521e4da75b8a1
Jul 23 15:50:11.233158 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.232198    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d8984d499-4m7rc"]
Jul 23 15:50:11.237955 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:50:11.237908    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c1bc9a7_99f1_4dba_b6b1_cc797948f674.slice/crio-c1beb3688df1536e4e3d32a93676e66256fcd45cff81e5d56abbb65f6a404160 WatchSource:0}: Error finding container c1beb3688df1536e4e3d32a93676e66256fcd45cff81e5d56abbb65f6a404160: Status 404 returned error can't find the container with id c1beb3688df1536e4e3d32a93676e66256fcd45cff81e5d56abbb65f6a404160
Jul 23 15:50:11.254455 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.254106    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"]
Jul 23 15:50:11.271160 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.271128    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"]
Jul 23 15:50:11.271346 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.271175    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d788c457-gww6n"]
Jul 23 15:50:11.271425 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.271399    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"
Jul 23 15:50:11.278569 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.278075    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-c2fxc\""
Jul 23 15:50:11.278569 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.278223    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 23 15:50:11.284301 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.284009    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.301602 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.301525    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d788c457-gww6n"]
Jul 23 15:50:11.404154 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.404125    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76785b994f-zdt2n"]
Jul 23 15:50:11.406285 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:50:11.406252    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode735cb6a_9f17_4603_9ac3_06f64f8d88c6.slice/crio-ea9c4141ae972d9afb01ee2f255031a22d03c344508f2418fbe4743955f12a22 WatchSource:0}: Error finding container ea9c4141ae972d9afb01ee2f255031a22d03c344508f2418fbe4743955f12a22: Status 404 returned error can't find the container with id ea9c4141ae972d9afb01ee2f255031a22d03c344508f2418fbe4743955f12a22
Jul 23 15:50:11.433487 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.433107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcdfv\" (UniqueName: \"kubernetes.io/projected/75cf7d4c-9cec-4f95-ba6a-9934fbb8221e-kube-api-access-xcdfv\") pod \"monitoring-plugin-7b7b7bc8bf-dmcs8\" (UID: \"75cf7d4c-9cec-4f95-ba6a-9934fbb8221e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"
Jul 23 15:50:11.433487 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.433183    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-oauth-config\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.433487 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.433272    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-console-config\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.433487 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.433305    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-service-ca\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.433487 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.433343    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47qd5\" (UniqueName: \"kubernetes.io/projected/d8680750-4437-46fe-bc00-4c65413611ce-kube-api-access-47qd5\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.433487 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.433399    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75cf7d4c-9cec-4f95-ba6a-9934fbb8221e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-dmcs8\" (UID: \"75cf7d4c-9cec-4f95-ba6a-9934fbb8221e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"
Jul 23 15:50:11.433905 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.433465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-serving-cert\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.433905 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.433530    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-oauth-serving-cert\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.433905 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.433624    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-trusted-ca-bundle\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.534904 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.534337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-trusted-ca-bundle\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.534904 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.534406    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcdfv\" (UniqueName: \"kubernetes.io/projected/75cf7d4c-9cec-4f95-ba6a-9934fbb8221e-kube-api-access-xcdfv\") pod \"monitoring-plugin-7b7b7bc8bf-dmcs8\" (UID: \"75cf7d4c-9cec-4f95-ba6a-9934fbb8221e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"
Jul 23 15:50:11.534904 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.534436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-oauth-config\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.534904 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.534506    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-console-config\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.534904 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.534537    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-service-ca\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.534904 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.534570    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47qd5\" (UniqueName: \"kubernetes.io/projected/d8680750-4437-46fe-bc00-4c65413611ce-kube-api-access-47qd5\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.534904 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.534597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75cf7d4c-9cec-4f95-ba6a-9934fbb8221e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-dmcs8\" (UID: \"75cf7d4c-9cec-4f95-ba6a-9934fbb8221e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"
Jul 23 15:50:11.534904 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.534626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-serving-cert\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.534904 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.534651    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-oauth-serving-cert\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.536972 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.536923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-trusted-ca-bundle\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.536972 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.536960    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-oauth-serving-cert\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.538076 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.538032    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-service-ca\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.538470 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.538433    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-console-config\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.539545 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.539502    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75cf7d4c-9cec-4f95-ba6a-9934fbb8221e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-dmcs8\" (UID: \"75cf7d4c-9cec-4f95-ba6a-9934fbb8221e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"
Jul 23 15:50:11.539845 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.539776    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-oauth-config\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.540573 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.540533    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-serving-cert\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.543947 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.543905    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47qd5\" (UniqueName: \"kubernetes.io/projected/d8680750-4437-46fe-bc00-4c65413611ce-kube-api-access-47qd5\") pod \"console-6d788c457-gww6n\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") " pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.544445 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.544424    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcdfv\" (UniqueName: \"kubernetes.io/projected/75cf7d4c-9cec-4f95-ba6a-9934fbb8221e-kube-api-access-xcdfv\") pod \"monitoring-plugin-7b7b7bc8bf-dmcs8\" (UID: \"75cf7d4c-9cec-4f95-ba6a-9934fbb8221e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"
Jul 23 15:50:11.586702 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.586664    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"
Jul 23 15:50:11.601074 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.600988    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:11.691826 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.691766    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sc4r8" event={"ID":"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7","Type":"ContainerStarted","Data":"e558830b60aac1a5818ec4004bfe6654c07640a408b3b1ad9d3911114b8db3dc"}
Jul 23 15:50:11.694602 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.694498    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cnjpz" event={"ID":"222a0a15-7ded-4543-94e0-21ddcd1bcf3d","Type":"ContainerStarted","Data":"87499a12016109fc4b66f5a5966903a9aa4a0259357b1c69a4a8ae60ebf994ca"}
Jul 23 15:50:11.694602 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.694554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cnjpz" event={"ID":"222a0a15-7ded-4543-94e0-21ddcd1bcf3d","Type":"ContainerStarted","Data":"66f2d43232de8de01b2e9385eec5045ee0e85f16a1cba385848d663b5a48c764"}
Jul 23 15:50:11.696166 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.696136    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerStarted","Data":"c365e3db3aa8fc813941e68ca097664cfebdd5a27e29760932d521e4da75b8a1"}
Jul 23 15:50:11.698083 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.698030    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76785b994f-zdt2n" event={"ID":"e735cb6a-9f17-4603-9ac3-06f64f8d88c6","Type":"ContainerStarted","Data":"ea9c4141ae972d9afb01ee2f255031a22d03c344508f2418fbe4743955f12a22"}
Jul 23 15:50:11.702143 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.701634    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zhhqf" event={"ID":"1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441","Type":"ContainerStarted","Data":"13e081d6429ba348ed9eb677c4cc4f097ebb818b738756f1f590eb3d632f066b"}
Jul 23 15:50:11.702143 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.701773    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:50:11.704225 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.703716    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sl96g" event={"ID":"9b20f8df-5ab7-4534-8ae3-f51228f128b8","Type":"ContainerStarted","Data":"b143835113a6d683d7ecd4e3856790d99b4ee87589ee075155bf749c3be320f7"}
Jul 23 15:50:11.707459 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.707197    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk" event={"ID":"ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e","Type":"ContainerStarted","Data":"ece23ee2aaa80c0de544559b75236d089d6aeb71277285b8a6f4776e20955bef"}
Jul 23 15:50:11.709393 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.709271    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc" event={"ID":"5c1bc9a7-99f1-4dba-b6b1-cc797948f674","Type":"ContainerStarted","Data":"c1beb3688df1536e4e3d32a93676e66256fcd45cff81e5d56abbb65f6a404160"}
Jul 23 15:50:11.713283 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.713232    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cnjpz" podStartSLOduration=68.429533037 podStartE2EDuration="1m14.713213483s" totalImagesPullingTime="6.283680446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:11.710868102 +0000 UTC m=+74.915235495" watchObservedRunningTime="2026-07-23 15:50:11.713213483 +0000 UTC m=+74.917580877"
Jul 23 15:50:11.729992 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.729931    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zhhqf" podStartSLOduration=68.183670555 podStartE2EDuration="1m14.729906537s" totalImagesPullingTime="6.546235982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:11.727642394 +0000 UTC m=+74.932009788" watchObservedRunningTime="2026-07-23 15:50:11.729906537 +0000 UTC m=+74.934273932"
Jul 23 15:50:11.757155 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.754313    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gq2vk" podStartSLOduration=1.998027663 podStartE2EDuration="5.754294683s" totalImagesPullingTime="3.75626702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:11.747018362 +0000 UTC m=+74.951385756" watchObservedRunningTime="2026-07-23 15:50:11.754294683 +0000 UTC m=+74.958662077"
Jul 23 15:50:11.768204 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.767157    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sl96g" podStartSLOduration=68.191670493 podStartE2EDuration="1m14.767134697s" totalImagesPullingTime="6.575464204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:11.765407539 +0000 UTC m=+74.969774933" watchObservedRunningTime="2026-07-23 15:50:11.767134697 +0000 UTC m=+74.971502091"
Jul 23 15:50:11.781426 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.781397    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"]
Jul 23 15:50:11.812551 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:11.812518    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d788c457-gww6n"]
Jul 23 15:50:12.113061 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:50:12.112977    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75cf7d4c_9cec_4f95_ba6a_9934fbb8221e.slice/crio-4eb54e9a8a0e0956148177c6e6c067d793e3385563641b9c0fc3c5a7e8396ee3 WatchSource:0}: Error finding container 4eb54e9a8a0e0956148177c6e6c067d793e3385563641b9c0fc3c5a7e8396ee3: Status 404 returned error can't find the container with id 4eb54e9a8a0e0956148177c6e6c067d793e3385563641b9c0fc3c5a7e8396ee3
Jul 23 15:50:12.114115 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:50:12.114089    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8680750_4437_46fe_bc00_4c65413611ce.slice/crio-2f61ecf561492bd6cd4331e7604674f0c5c413cb9bd410a56a96249d9c6c6ce5 WatchSource:0}: Error finding container 2f61ecf561492bd6cd4331e7604674f0c5c413cb9bd410a56a96249d9c6c6ce5: Status 404 returned error can't find the container with id 2f61ecf561492bd6cd4331e7604674f0c5c413cb9bd410a56a96249d9c6c6ce5
Jul 23 15:50:12.719665 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.719616    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d788c457-gww6n" event={"ID":"d8680750-4437-46fe-bc00-4c65413611ce","Type":"ContainerStarted","Data":"2f61ecf561492bd6cd4331e7604674f0c5c413cb9bd410a56a96249d9c6c6ce5"}
Jul 23 15:50:12.721329 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.721302    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8" event={"ID":"75cf7d4c-9cec-4f95-ba6a-9934fbb8221e","Type":"ContainerStarted","Data":"4eb54e9a8a0e0956148177c6e6c067d793e3385563641b9c0fc3c5a7e8396ee3"}
Jul 23 15:50:12.783889 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.782928    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 15:50:12.807163 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.805322    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.812260 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.811915    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 23 15:50:12.817073 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.813320    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 23 15:50:12.817073 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.813759    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 23 15:50:12.817073 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.814091    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 23 15:50:12.817073 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.814754    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 23 15:50:12.817073 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.815211    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 23 15:50:12.817073 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.815395    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 23 15:50:12.817073 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.816638    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 23 15:50:12.817542 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.817499    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-67zb2\""
Jul 23 15:50:12.817833 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.817732    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 23 15:50:12.821362 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.820547    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 23 15:50:12.825440 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.824905    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 23 15:50:12.834636 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.825663    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9v93cbs5vmt17\""
Jul 23 15:50:12.834636 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.826287    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 23 15:50:12.834636 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.829290    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 23 15:50:12.834636 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.833585    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 15:50:12.839463 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.838575    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.946686    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.946743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.946784    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpjg5\" (UniqueName: \"kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-kube-api-access-qpjg5\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.946837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.946873    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.946927    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-config-out\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.946956    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.946982    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.947006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.947030    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.947063    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.947095    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.947128    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.947155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.947180    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.947344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.947221    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.949463 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.947247    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-web-config\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.949463 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.949246    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.949463 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.949314    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-config\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:12.949463 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:12.949346    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.051716 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.050167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.051716 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.050234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-config-out\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.051716 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.050261    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.051716 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.050285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.051716 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.050313    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.054997 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.054705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.054997 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.054782    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.054997 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.054839    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.054997 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.054878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.054997 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.054883    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.054997 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.054903    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.055405 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.055251    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.055405 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.055311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.055405 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.055341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-web-config\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.055405 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.055371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.055604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.055426    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-config\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.055604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.055459    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.055604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.055504    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.055604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.055532    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.055604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.055532    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.055604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.055583    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpjg5\" (UniqueName: \"kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-kube-api-access-qpjg5\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.055925 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.055620    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.062937 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.058880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.062937 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.060049    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-config\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.063135 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.063016    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.065837 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.065759    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.069180 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.068108    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.069180 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.068129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-web-config\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.069180 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.068544    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.069979 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.069930    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.070573 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.070525    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.070673 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.070583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.071791 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.071748    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.080909 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.080843    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpjg5\" (UniqueName: \"kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-kube-api-access-qpjg5\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.081045 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.081025    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.081288 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.081238    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-config-out\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.085528 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.081614    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.085528 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.083168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.085528 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.083276    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.086867 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.086789    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.185453 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.185235    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:13.434034 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.434000    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 15:50:13.729043 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.728521    2562 generic.go:356] "Generic (PLEG): container finished" podID="35f95106-7910-4fd8-9f81-2f9c2ac2d0f7" containerID="83d602b476491c8e7459aed5f24bdca8c84415b4bfaff1cbff331adfc48ca249" exitCode=0
Jul 23 15:50:13.729043 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.728633    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sc4r8" event={"ID":"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7","Type":"ContainerDied","Data":"83d602b476491c8e7459aed5f24bdca8c84415b4bfaff1cbff331adfc48ca249"}
Jul 23 15:50:13.734437 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.733719    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857" event={"ID":"e46ca500-8035-4e99-a59e-60270d18e78a","Type":"ContainerStarted","Data":"51ab3b90dea58a0f4057bf6eac07bdb20c4a4ec60e2eea7e84fa9cd82a8c1b25"}
Jul 23 15:50:13.734437 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.733787    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857" event={"ID":"e46ca500-8035-4e99-a59e-60270d18e78a","Type":"ContainerStarted","Data":"bf6c68d17f3480fedbb54afbce9db45ff593ff40e27566a8cf88f70c0015c020"}
Jul 23 15:50:13.734437 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.733835    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857" event={"ID":"e46ca500-8035-4e99-a59e-60270d18e78a","Type":"ContainerStarted","Data":"84653a719c1d63b487aeab979fbaab37e3f699d3a0978c333d5b37402bd3811f"}
Jul 23 15:50:13.736245 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.736183    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d788c457-gww6n" event={"ID":"d8680750-4437-46fe-bc00-4c65413611ce","Type":"ContainerStarted","Data":"f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd"}
Jul 23 15:50:13.738941 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.738905    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerStarted","Data":"9eaced9980065496cd1ddb63e94e201d234f60bea61ad1a5d40056c5d0881e87"}
Jul 23 15:50:13.770660 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.770556    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-6b857" podStartSLOduration=4.184783806 podStartE2EDuration="7.77053734s" totalImagesPullingTime="3.585753534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:13.768968192 +0000 UTC m=+76.973335586" watchObservedRunningTime="2026-07-23 15:50:13.77053734 +0000 UTC m=+76.974904766"
Jul 23 15:50:13.789074 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:13.788934    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d788c457-gww6n" podStartSLOduration=2.788910333 podStartE2EDuration="2.788910333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:13.787609155 +0000 UTC m=+76.991976560" watchObservedRunningTime="2026-07-23 15:50:13.788910333 +0000 UTC m=+76.993277727"
Jul 23 15:50:16.756700 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.756656    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerStarted","Data":"cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203"}
Jul 23 15:50:16.758640 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.758597    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76785b994f-zdt2n" event={"ID":"e735cb6a-9f17-4603-9ac3-06f64f8d88c6","Type":"ContainerStarted","Data":"db52e1f29ab69f2a32cc752e0da111117ae20324e385ac7ec23dbef5b9d483e0"}
Jul 23 15:50:16.760245 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.760195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc" event={"ID":"5c1bc9a7-99f1-4dba-b6b1-cc797948f674","Type":"ContainerStarted","Data":"efd11c43e971249095c6df9e64881972002638d15df7110924f2e70b9eaf849c"}
Jul 23 15:50:16.762508 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.762479    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerStarted","Data":"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"}
Jul 23 15:50:16.764658 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.764600    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8" event={"ID":"75cf7d4c-9cec-4f95-ba6a-9934fbb8221e","Type":"ContainerStarted","Data":"6869615916ce4e7d4a585ede6a4211dcf88776cf8d15855cda93ff82b1b08fbd"}
Jul 23 15:50:16.766265 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.766221    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"
Jul 23 15:50:16.768387 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.768311    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sc4r8" event={"ID":"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7","Type":"ContainerStarted","Data":"7e0fb09e3bf0493a0b7816d1b54dc683f1820273d6103b8988851a7da59ae50d"}
Jul 23 15:50:16.768387 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.768344    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sc4r8" event={"ID":"35f95106-7910-4fd8-9f81-2f9c2ac2d0f7","Type":"ContainerStarted","Data":"d8c029398e88a23d8e58160fa934c095e57a662220d4803929513f0b7cd046ef"}
Jul 23 15:50:16.772674 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.772625    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8"
Jul 23 15:50:16.785271 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.785209    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dmcs8" podStartSLOduration=1.5063023850000001 podStartE2EDuration="5.785186013s" totalImagesPullingTime="4.278883628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:16.783976873 +0000 UTC m=+79.988344266" watchObservedRunningTime="2026-07-23 15:50:16.785186013 +0000 UTC m=+79.989553401"
Jul 23 15:50:16.807711 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.807650    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-76785b994f-zdt2n" podStartSLOduration=1.822342246 podStartE2EDuration="6.807622788s" totalImagesPullingTime="4.985280542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:16.805557785 +0000 UTC m=+80.009925178" watchObservedRunningTime="2026-07-23 15:50:16.807622788 +0000 UTC m=+80.011990182"
Jul 23 15:50:16.871375 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:16.871304    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sc4r8" podStartSLOduration=8.966709735 podStartE2EDuration="10.871282278s" totalImagesPullingTime="1.742263247s" totalInitContainerRuntime="162.309296ms" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:16.870438738 +0000 UTC m=+80.074806132" watchObservedRunningTime="2026-07-23 15:50:16.871282278 +0000 UTC m=+80.075649713"
Jul 23 15:50:17.774926 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:17.774890    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc" event={"ID":"5c1bc9a7-99f1-4dba-b6b1-cc797948f674","Type":"ContainerStarted","Data":"aee2e64a58d0b5f08ade6b4c97e496b2f11a594ce7cabf2dd31b729114591071"}
Jul 23 15:50:17.775451 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:17.774967    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc" event={"ID":"5c1bc9a7-99f1-4dba-b6b1-cc797948f674","Type":"ContainerStarted","Data":"a639883e0c15c31811082a676cdbb6375df00257ed38e528316490f69c4cf764"}
Jul 23 15:50:17.776470 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:17.776435    2562 generic.go:356] "Generic (PLEG): container finished" podID="bf786fed-02bb-473b-873a-b06582337668" containerID="2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e" exitCode=0
Jul 23 15:50:17.776616 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:17.776521    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerDied","Data":"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"}
Jul 23 15:50:17.778186 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:17.778168    2562 generic.go:356] "Generic (PLEG): container finished" podID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerID="cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203" exitCode=0
Jul 23 15:50:17.778295 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:17.778275    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerDied","Data":"cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203"}
Jul 23 15:50:19.795290 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:19.795194    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc" event={"ID":"5c1bc9a7-99f1-4dba-b6b1-cc797948f674","Type":"ContainerStarted","Data":"f23eb057fc0aee14882899320faffdf857b1107700d7624c9595e7326e71a3bf"}
Jul 23 15:50:19.795751 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:19.795329    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc" event={"ID":"5c1bc9a7-99f1-4dba-b6b1-cc797948f674","Type":"ContainerStarted","Data":"5c81b61df3500163fb961ecaf257a3c9af3f3666a406e45f70c4451720adc510"}
Jul 23 15:50:19.795751 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:19.795378    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc" event={"ID":"5c1bc9a7-99f1-4dba-b6b1-cc797948f674","Type":"ContainerStarted","Data":"99ea0c540f37c3d51fa94f76d586ee09b4ed91f5cfac2b32d8611619d03d1e8c"}
Jul 23 15:50:19.795751 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:19.795694    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:21.601648 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:21.601605    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:21.602200 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:21.601694    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:21.607345 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:21.607318    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:21.626333 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:21.626274    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc" podStartSLOduration=6.373126526 podStartE2EDuration="13.626251489s" totalImagesPullingTime="7.253124963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:19.83515922 +0000 UTC m=+83.039526625" watchObservedRunningTime="2026-07-23 15:50:21.626251489 +0000 UTC m=+84.830618884"
Jul 23 15:50:21.810154 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:21.810122    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:50:21.855891 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:21.855816    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cf55466c-fjr89"]
Jul 23 15:50:22.813914 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:22.813873    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerStarted","Data":"a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1"}
Jul 23 15:50:22.814291 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:22.814004    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerStarted","Data":"0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30"}
Jul 23 15:50:22.814291 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:22.814058    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerStarted","Data":"d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a"}
Jul 23 15:50:22.817389 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:22.817363    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerStarted","Data":"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"}
Jul 23 15:50:22.817460 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:22.817450    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerStarted","Data":"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"}
Jul 23 15:50:22.817523 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:22.817494    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerStarted","Data":"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"}
Jul 23 15:50:23.825352 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:23.825315    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerStarted","Data":"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"}
Jul 23 15:50:23.825933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:23.825401    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerStarted","Data":"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"}
Jul 23 15:50:23.825933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:23.825454    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerStarted","Data":"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"}
Jul 23 15:50:23.828473 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:23.828445    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerStarted","Data":"2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333"}
Jul 23 15:50:23.828611 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:23.828504    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerStarted","Data":"3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698"}
Jul 23 15:50:23.828611 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:23.828537    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerStarted","Data":"01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4"}
Jul 23 15:50:23.862505 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:23.862437    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.067252493 podStartE2EDuration="11.862415165s" totalImagesPullingTime="7.751466034s" totalInitContainerRuntime="43.696638ms" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:23.860097058 +0000 UTC m=+87.064464451" watchObservedRunningTime="2026-07-23 15:50:23.862415165 +0000 UTC m=+87.066782559"
Jul 23 15:50:23.891174 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:23.891117    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=7.144758655 podStartE2EDuration="16.891099115s" totalImagesPullingTime="9.711193441s" totalInitContainerRuntime="35.147019ms" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:23.8889683 +0000 UTC m=+87.093335717" watchObservedRunningTime="2026-07-23 15:50:23.891099115 +0000 UTC m=+87.095466508"
Jul 23 15:50:25.806821 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:25.806781    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6d8984d499-4m7rc"
Jul 23 15:50:28.185588 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:28.185549    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:50:31.197739 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:31.197701    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:31.198247 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:31.197787    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:31.776491 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:31.776431    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-796cffdbf5-gj9g5" podUID="a7df6af1-0ff1-4586-b3a3-e432b389f372" containerName="console" containerID="cri-o://230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4" gracePeriod=15
Jul 23 15:50:32.057388 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.057351    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:50:32.140336 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.140298    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-serving-cert\") pod \"a7df6af1-0ff1-4586-b3a3-e432b389f372\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") "
Jul 23 15:50:32.140512 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.140352    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-service-ca\") pod \"a7df6af1-0ff1-4586-b3a3-e432b389f372\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") "
Jul 23 15:50:32.140512 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.140384    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-oauth-config\") pod \"a7df6af1-0ff1-4586-b3a3-e432b389f372\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") "
Jul 23 15:50:32.140512 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.140404    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-oauth-serving-cert\") pod \"a7df6af1-0ff1-4586-b3a3-e432b389f372\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") "
Jul 23 15:50:32.140512 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.140481    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwsbn\" (UniqueName: \"kubernetes.io/projected/a7df6af1-0ff1-4586-b3a3-e432b389f372-kube-api-access-pwsbn\") pod \"a7df6af1-0ff1-4586-b3a3-e432b389f372\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") "
Jul 23 15:50:32.140680 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.140527    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-config\") pod \"a7df6af1-0ff1-4586-b3a3-e432b389f372\" (UID: \"a7df6af1-0ff1-4586-b3a3-e432b389f372\") "
Jul 23 15:50:32.140926 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.140898    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-service-ca" (OuterVolumeSpecName: "service-ca") pod "a7df6af1-0ff1-4586-b3a3-e432b389f372" (UID: "a7df6af1-0ff1-4586-b3a3-e432b389f372"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:50:32.141025 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.140941    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a7df6af1-0ff1-4586-b3a3-e432b389f372" (UID: "a7df6af1-0ff1-4586-b3a3-e432b389f372"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:50:32.141025 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.140970    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-config" (OuterVolumeSpecName: "console-config") pod "a7df6af1-0ff1-4586-b3a3-e432b389f372" (UID: "a7df6af1-0ff1-4586-b3a3-e432b389f372"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:50:32.142788 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.142762    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a7df6af1-0ff1-4586-b3a3-e432b389f372" (UID: "a7df6af1-0ff1-4586-b3a3-e432b389f372"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:50:32.142883 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.142782    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7df6af1-0ff1-4586-b3a3-e432b389f372-kube-api-access-pwsbn" (OuterVolumeSpecName: "kube-api-access-pwsbn") pod "a7df6af1-0ff1-4586-b3a3-e432b389f372" (UID: "a7df6af1-0ff1-4586-b3a3-e432b389f372"). InnerVolumeSpecName "kube-api-access-pwsbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 15:50:32.142883 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.142859    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a7df6af1-0ff1-4586-b3a3-e432b389f372" (UID: "a7df6af1-0ff1-4586-b3a3-e432b389f372"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:50:32.148781 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.148757    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-695p5_6e24b669-70a0-4855-8cbe-a476295a950a/serve-healthcheck-canary/0.log"
Jul 23 15:50:32.241934 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.241901    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:32.241934 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.241927    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:32.241934 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.241936    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-oauth-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:32.241934 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.241945    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-oauth-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:32.242412 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.241954    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwsbn\" (UniqueName: \"kubernetes.io/projected/a7df6af1-0ff1-4586-b3a3-e432b389f372-kube-api-access-pwsbn\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:32.242412 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.241962    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7df6af1-0ff1-4586-b3a3-e432b389f372-console-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:32.867641 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.867605    2562 generic.go:356] "Generic (PLEG): container finished" podID="a7df6af1-0ff1-4586-b3a3-e432b389f372" containerID="230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4" exitCode=0
Jul 23 15:50:32.867838 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.867655    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796cffdbf5-gj9g5"
Jul 23 15:50:32.867838 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.867654    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796cffdbf5-gj9g5" event={"ID":"a7df6af1-0ff1-4586-b3a3-e432b389f372","Type":"ContainerDied","Data":"230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4"}
Jul 23 15:50:32.867838 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.867778    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796cffdbf5-gj9g5" event={"ID":"a7df6af1-0ff1-4586-b3a3-e432b389f372","Type":"ContainerDied","Data":"b3f1363a64d229041b0f68135919bd68720a791ff6f9343a5972cb8ebf613d5f"}
Jul 23 15:50:32.867838 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.867824    2562 scope.go:117] "RemoveContainer" containerID="230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4"
Jul 23 15:50:32.876974 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.876952    2562 scope.go:117] "RemoveContainer" containerID="230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4"
Jul 23 15:50:32.877257 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:50:32.877237    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4\": container with ID starting with 230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4 not found: ID does not exist" containerID="230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4"
Jul 23 15:50:32.877335 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.877273    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4"} err="failed to get container status \"230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4\": rpc error: code = NotFound desc = could not find container \"230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4\": container with ID starting with 230ab06925b2e64c65ad04e39a4347b6ab818a9e2b19c26c2660073fabe0f1c4 not found: ID does not exist"
Jul 23 15:50:32.890025 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.889996    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-796cffdbf5-gj9g5"]
Jul 23 15:50:32.893996 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:32.893974    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-796cffdbf5-gj9g5"]
Jul 23 15:50:33.255517 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:33.255483    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7df6af1-0ff1-4586-b3a3-e432b389f372" path="/var/lib/kubelet/pods/a7df6af1-0ff1-4586-b3a3-e432b389f372/volumes"
Jul 23 15:50:42.724280 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:42.724250    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zhhqf"
Jul 23 15:50:46.883191 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:46.883126    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cf55466c-fjr89" podUID="22b77565-5841-4ebf-8c58-45330e3347fc" containerName="console" containerID="cri-o://80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed" gracePeriod=15
Jul 23 15:50:47.680648 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.680613    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:50:47.785864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.785752    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-service-ca\") pod \"22b77565-5841-4ebf-8c58-45330e3347fc\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") "
Jul 23 15:50:47.785864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.785797    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-oauth-serving-cert\") pod \"22b77565-5841-4ebf-8c58-45330e3347fc\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") "
Jul 23 15:50:47.786096 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.785945    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-console-config\") pod \"22b77565-5841-4ebf-8c58-45330e3347fc\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") "
Jul 23 15:50:47.786096 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.786039    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbf4b\" (UniqueName: \"kubernetes.io/projected/22b77565-5841-4ebf-8c58-45330e3347fc-kube-api-access-nbf4b\") pod \"22b77565-5841-4ebf-8c58-45330e3347fc\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") "
Jul 23 15:50:47.786096 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.786063    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-oauth-config\") pod \"22b77565-5841-4ebf-8c58-45330e3347fc\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") "
Jul 23 15:50:47.786258 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.786106    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-trusted-ca-bundle\") pod \"22b77565-5841-4ebf-8c58-45330e3347fc\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") "
Jul 23 15:50:47.786258 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.786136    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-serving-cert\") pod \"22b77565-5841-4ebf-8c58-45330e3347fc\" (UID: \"22b77565-5841-4ebf-8c58-45330e3347fc\") "
Jul 23 15:50:47.786258 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.786193    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-service-ca" (OuterVolumeSpecName: "service-ca") pod "22b77565-5841-4ebf-8c58-45330e3347fc" (UID: "22b77565-5841-4ebf-8c58-45330e3347fc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:50:47.786534 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.786502    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "22b77565-5841-4ebf-8c58-45330e3347fc" (UID: "22b77565-5841-4ebf-8c58-45330e3347fc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:50:47.786613 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.786511    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:47.786666 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.786605    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-console-config" (OuterVolumeSpecName: "console-config") pod "22b77565-5841-4ebf-8c58-45330e3347fc" (UID: "22b77565-5841-4ebf-8c58-45330e3347fc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:50:47.788372 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.788335    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "22b77565-5841-4ebf-8c58-45330e3347fc" (UID: "22b77565-5841-4ebf-8c58-45330e3347fc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:50:47.788787 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.788764    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22b77565-5841-4ebf-8c58-45330e3347fc-kube-api-access-nbf4b" (OuterVolumeSpecName: "kube-api-access-nbf4b") pod "22b77565-5841-4ebf-8c58-45330e3347fc" (UID: "22b77565-5841-4ebf-8c58-45330e3347fc"). InnerVolumeSpecName "kube-api-access-nbf4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 15:50:47.788893 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.788769    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "22b77565-5841-4ebf-8c58-45330e3347fc" (UID: "22b77565-5841-4ebf-8c58-45330e3347fc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:50:47.793395 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.793368    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "22b77565-5841-4ebf-8c58-45330e3347fc" (UID: "22b77565-5841-4ebf-8c58-45330e3347fc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:50:47.887366 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.887333    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbf4b\" (UniqueName: \"kubernetes.io/projected/22b77565-5841-4ebf-8c58-45330e3347fc-kube-api-access-nbf4b\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:47.887366 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.887362    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-oauth-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:47.887366 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.887372    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-trusted-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:47.887834 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.887381    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b77565-5841-4ebf-8c58-45330e3347fc-console-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:47.887834 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.887390    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-oauth-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:47.887834 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.887399    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22b77565-5841-4ebf-8c58-45330e3347fc-console-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:50:47.942144 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.942111    2562 generic.go:356] "Generic (PLEG): container finished" podID="22b77565-5841-4ebf-8c58-45330e3347fc" containerID="80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed" exitCode=0
Jul 23 15:50:47.942328 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.942156    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf55466c-fjr89"
Jul 23 15:50:47.942328 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.942204    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf55466c-fjr89" event={"ID":"22b77565-5841-4ebf-8c58-45330e3347fc","Type":"ContainerDied","Data":"80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed"}
Jul 23 15:50:47.942328 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.942265    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf55466c-fjr89" event={"ID":"22b77565-5841-4ebf-8c58-45330e3347fc","Type":"ContainerDied","Data":"db0a20e0c26912b10d7a8f5f4fff44fb41732cf91042406f5d7a92555ee6b755"}
Jul 23 15:50:47.942328 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.942293    2562 scope.go:117] "RemoveContainer" containerID="80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed"
Jul 23 15:50:47.952235 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.952218    2562 scope.go:117] "RemoveContainer" containerID="80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed"
Jul 23 15:50:47.952503 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:50:47.952486    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed\": container with ID starting with 80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed not found: ID does not exist" containerID="80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed"
Jul 23 15:50:47.952569 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.952512    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed"} err="failed to get container status \"80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed\": rpc error: code = NotFound desc = could not find container \"80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed\": container with ID starting with 80b7d06e9ab3fdac9e0ed0722887893d3186aca6d102b0a839004d6112c3a1ed not found: ID does not exist"
Jul 23 15:50:47.971950 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.971695    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cf55466c-fjr89"]
Jul 23 15:50:47.973848 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:47.973820    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cf55466c-fjr89"]
Jul 23 15:50:49.253316 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:49.253272    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22b77565-5841-4ebf-8c58-45330e3347fc" path="/var/lib/kubelet/pods/22b77565-5841-4ebf-8c58-45330e3347fc/volumes"
Jul 23 15:50:51.203944 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:51.203918    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:50:51.207931 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:50:51.207905    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-76785b994f-zdt2n"
Jul 23 15:51:13.186182 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:13.186143    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:13.204873 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:13.204843    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:14.064949 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:14.064919    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:26.815040 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:26.815004    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 15:51:26.815579 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:26.815362    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="alertmanager" containerID="cri-o://d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a" gracePeriod=120
Jul 23 15:51:26.815579 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:26.815414    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy-metric" containerID="cri-o://3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698" gracePeriod=120
Jul 23 15:51:26.815579 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:26.815441    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy-web" containerID="cri-o://a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1" gracePeriod=120
Jul 23 15:51:26.815579 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:26.815441    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy" containerID="cri-o://01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4" gracePeriod=120
Jul 23 15:51:26.815579 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:26.815498    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="prom-label-proxy" containerID="cri-o://2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333" gracePeriod=120
Jul 23 15:51:26.815579 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:26.815459    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="config-reloader" containerID="cri-o://0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30" gracePeriod=120
Jul 23 15:51:27.103180 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:27.103097    2562 generic.go:356] "Generic (PLEG): container finished" podID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerID="2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333" exitCode=0
Jul 23 15:51:27.103180 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:27.103121    2562 generic.go:356] "Generic (PLEG): container finished" podID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerID="3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698" exitCode=0
Jul 23 15:51:27.103180 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:27.103127    2562 generic.go:356] "Generic (PLEG): container finished" podID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerID="01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4" exitCode=0
Jul 23 15:51:27.103180 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:27.103133    2562 generic.go:356] "Generic (PLEG): container finished" podID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerID="0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30" exitCode=0
Jul 23 15:51:27.103180 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:27.103138    2562 generic.go:356] "Generic (PLEG): container finished" podID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerID="d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a" exitCode=0
Jul 23 15:51:27.103180 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:27.103171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerDied","Data":"2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333"}
Jul 23 15:51:27.103492 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:27.103236    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerDied","Data":"3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698"}
Jul 23 15:51:27.103492 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:27.103264    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerDied","Data":"01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4"}
Jul 23 15:51:27.103492 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:27.103287    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerDied","Data":"0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30"}
Jul 23 15:51:27.103492 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:27.103310    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerDied","Data":"d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a"}
Jul 23 15:51:28.060168 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.060145    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.114368 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.114336    2562 generic.go:356] "Generic (PLEG): container finished" podID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerID="a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1" exitCode=0
Jul 23 15:51:28.114555 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.114401    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerDied","Data":"a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1"}
Jul 23 15:51:28.114555 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.114409    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.114555 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.114458    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa2732c-ace1-4462-abde-fb0ad81b0d4d","Type":"ContainerDied","Data":"c365e3db3aa8fc813941e68ca097664cfebdd5a27e29760932d521e4da75b8a1"}
Jul 23 15:51:28.114555 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.114491    2562 scope.go:117] "RemoveContainer" containerID="2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333"
Jul 23 15:51:28.123098 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.122956    2562 scope.go:117] "RemoveContainer" containerID="3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698"
Jul 23 15:51:28.130699 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.130682    2562 scope.go:117] "RemoveContainer" containerID="01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4"
Jul 23 15:51:28.137443 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.137419    2562 scope.go:117] "RemoveContainer" containerID="a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1"
Jul 23 15:51:28.143897 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.143870    2562 scope.go:117] "RemoveContainer" containerID="0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30"
Jul 23 15:51:28.145921 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.145902    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-out\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146025 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.145931    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-main-db\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146025 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.145956    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-web\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146025 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.145991    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-tls-assets\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146025 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146018    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146230 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146047    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-trusted-ca-bundle\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146230 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146077    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-volume\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146230 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146105    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146230 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146133    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rnr9\" (UniqueName: \"kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-kube-api-access-4rnr9\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146230 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146167    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-cluster-tls-config\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146230 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146208    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-main-tls\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146523 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146235    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-metrics-client-ca\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.146523 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146233    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 15:51:28.146523 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146272    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-web-config\") pod \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\" (UID: \"0aa2732c-ace1-4462-abde-fb0ad81b0d4d\") "
Jul 23 15:51:28.147144 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146911    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:51:28.147144 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.146923    2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-main-db\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.147144 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.147105    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:51:28.149421 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.149282    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-out" (OuterVolumeSpecName: "config-out") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 15:51:28.149421 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.149285    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-kube-api-access-4rnr9" (OuterVolumeSpecName: "kube-api-access-4rnr9") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "kube-api-access-4rnr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 15:51:28.149421 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.149366    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:28.150389 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.150354    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 15:51:28.150389 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.150370    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:28.150545 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.150475    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-volume" (OuterVolumeSpecName: "config-volume") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:28.150937 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.150908    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:28.151260 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.151235    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:28.154101 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.154077    2562 scope.go:117] "RemoveContainer" containerID="d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a"
Jul 23 15:51:28.154753 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.154725    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:28.161738 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.161713    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-web-config" (OuterVolumeSpecName: "web-config") pod "0aa2732c-ace1-4462-abde-fb0ad81b0d4d" (UID: "0aa2732c-ace1-4462-abde-fb0ad81b0d4d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:28.172263 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.172244    2562 scope.go:117] "RemoveContainer" containerID="cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203"
Jul 23 15:51:28.179484 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.179465    2562 scope.go:117] "RemoveContainer" containerID="2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333"
Jul 23 15:51:28.179751 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:28.179733    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333\": container with ID starting with 2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333 not found: ID does not exist" containerID="2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333"
Jul 23 15:51:28.179827 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.179761    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333"} err="failed to get container status \"2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333\": rpc error: code = NotFound desc = could not find container \"2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333\": container with ID starting with 2dae9f73d11e4373bcfd8877f79e41de44a073cdb62ca5fc5e5df3730424a333 not found: ID does not exist"
Jul 23 15:51:28.179827 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.179782    2562 scope.go:117] "RemoveContainer" containerID="3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698"
Jul 23 15:51:28.180063 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:28.180048    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698\": container with ID starting with 3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698 not found: ID does not exist" containerID="3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698"
Jul 23 15:51:28.180105 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.180069    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698"} err="failed to get container status \"3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698\": rpc error: code = NotFound desc = could not find container \"3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698\": container with ID starting with 3dbbcc763805898ef58a20a2b4ffa777148696edfde70b5d1cc2f37ffe6e5698 not found: ID does not exist"
Jul 23 15:51:28.180105 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.180084    2562 scope.go:117] "RemoveContainer" containerID="01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4"
Jul 23 15:51:28.180323 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:28.180305    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4\": container with ID starting with 01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4 not found: ID does not exist" containerID="01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4"
Jul 23 15:51:28.180380 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.180328    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4"} err="failed to get container status \"01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4\": rpc error: code = NotFound desc = could not find container \"01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4\": container with ID starting with 01ea555aad3717a885b12b29ebba0ef13a258354b87c63b3b9acc9d84fa2bfb4 not found: ID does not exist"
Jul 23 15:51:28.180380 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.180346    2562 scope.go:117] "RemoveContainer" containerID="a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1"
Jul 23 15:51:28.180575 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:28.180560    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1\": container with ID starting with a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1 not found: ID does not exist" containerID="a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1"
Jul 23 15:51:28.180615 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.180579    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1"} err="failed to get container status \"a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1\": rpc error: code = NotFound desc = could not find container \"a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1\": container with ID starting with a28975707d299b50a36158547c36dc6c81498d09c917a7e152c1714d845505a1 not found: ID does not exist"
Jul 23 15:51:28.180615 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.180594    2562 scope.go:117] "RemoveContainer" containerID="0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30"
Jul 23 15:51:28.180778 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:28.180760    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30\": container with ID starting with 0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30 not found: ID does not exist" containerID="0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30"
Jul 23 15:51:28.180833 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.180783    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30"} err="failed to get container status \"0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30\": rpc error: code = NotFound desc = could not find container \"0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30\": container with ID starting with 0ac3c9c4c3cb6e0091c11b055ae8e5b3b28542c68965cff7087be7a506145f30 not found: ID does not exist"
Jul 23 15:51:28.180833 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.180797    2562 scope.go:117] "RemoveContainer" containerID="d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a"
Jul 23 15:51:28.181010 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:28.180992    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a\": container with ID starting with d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a not found: ID does not exist" containerID="d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a"
Jul 23 15:51:28.181051 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.181012    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a"} err="failed to get container status \"d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a\": rpc error: code = NotFound desc = could not find container \"d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a\": container with ID starting with d13dc48e11604b88b64df548d75e627c5072d7424864c606bcdf09f713fe8d8a not found: ID does not exist"
Jul 23 15:51:28.181051 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.181025    2562 scope.go:117] "RemoveContainer" containerID="cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203"
Jul 23 15:51:28.181253 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:28.181237    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203\": container with ID starting with cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203 not found: ID does not exist" containerID="cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203"
Jul 23 15:51:28.181304 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.181256    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203"} err="failed to get container status \"cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203\": rpc error: code = NotFound desc = could not find container \"cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203\": container with ID starting with cd8d2aaa3ef48fe494fe7524c570ac93ae4ed818761cb6615d35f1aba8676203 not found: ID does not exist"
Jul 23 15:51:28.247601 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247564    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.247601 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247595    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.247864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247609    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.247864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247622    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.247864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247635    2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.247864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247646    2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-config-volume\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.247864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247657    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.247864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247669    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rnr9\" (UniqueName: \"kubernetes.io/projected/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-kube-api-access-4rnr9\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.247864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247680    2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-cluster-tls-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.247864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247691    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-secret-alertmanager-main-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.247864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247703    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-metrics-client-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.247864 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.247737    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aa2732c-ace1-4462-abde-fb0ad81b0d4d-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:28.438961 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.438926    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 15:51:28.442457 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.442430    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 15:51:28.469362 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469332    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 15:51:28.469651 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469639    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy-web"
Jul 23 15:51:28.469692 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469653    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy-web"
Jul 23 15:51:28.469692 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469666    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7df6af1-0ff1-4586-b3a3-e432b389f372" containerName="console"
Jul 23 15:51:28.469692 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469672    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7df6af1-0ff1-4586-b3a3-e432b389f372" containerName="console"
Jul 23 15:51:28.469692 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469678    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="alertmanager"
Jul 23 15:51:28.469692 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469684    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="alertmanager"
Jul 23 15:51:28.469692 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469692    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="config-reloader"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469698    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="config-reloader"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469704    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469709    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469722    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="init-config-reloader"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469728    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="init-config-reloader"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469734    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy-metric"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469739    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy-metric"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469748    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22b77565-5841-4ebf-8c58-45330e3347fc" containerName="console"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469753    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="22b77565-5841-4ebf-8c58-45330e3347fc" containerName="console"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469760    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="prom-label-proxy"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469764    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="prom-label-proxy"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469825    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy-web"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469832    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy-metric"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469838    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="22b77565-5841-4ebf-8c58-45330e3347fc" containerName="console"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469844    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7df6af1-0ff1-4586-b3a3-e432b389f372" containerName="console"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469849    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="alertmanager"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469854    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="config-reloader"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469861    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="prom-label-proxy"
Jul 23 15:51:28.469886 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.469867    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" containerName="kube-rbac-proxy"
Jul 23 15:51:28.475055 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.475034    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.477748 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.477721    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 23 15:51:28.477962 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.477880    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 23 15:51:28.477962 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.477896    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lz4sl\""
Jul 23 15:51:28.478128 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.477968    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 23 15:51:28.478128 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.478056    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 23 15:51:28.478233 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.478134    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 23 15:51:28.478233 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.478141    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 23 15:51:28.478233 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.478204    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 23 15:51:28.478372 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.478242    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 23 15:51:28.482970 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.482946    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 23 15:51:28.487298 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.487279    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 15:51:28.550911 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.550877    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.550911 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.550914    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.551177 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.550950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/05d673e3-59ac-4120-8a1a-90bd25c63823-tls-assets\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.551177 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.551027    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-web-config\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.551177 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.551066    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/05d673e3-59ac-4120-8a1a-90bd25c63823-config-out\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.551177 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.551093    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05d673e3-59ac-4120-8a1a-90bd25c63823-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.551177 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.551124    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.551177 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.551148    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkf62\" (UniqueName: \"kubernetes.io/projected/05d673e3-59ac-4120-8a1a-90bd25c63823-kube-api-access-fkf62\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.551386 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.551202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/05d673e3-59ac-4120-8a1a-90bd25c63823-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.551386 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.551226    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.551386 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.551257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-config-volume\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.551386 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.551282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05d673e3-59ac-4120-8a1a-90bd25c63823-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.551386 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.551298    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652094 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652094 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652332 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/05d673e3-59ac-4120-8a1a-90bd25c63823-tls-assets\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652332 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-web-config\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652332 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/05d673e3-59ac-4120-8a1a-90bd25c63823-config-out\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652332 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652222    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05d673e3-59ac-4120-8a1a-90bd25c63823-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652332 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652332 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkf62\" (UniqueName: \"kubernetes.io/projected/05d673e3-59ac-4120-8a1a-90bd25c63823-kube-api-access-fkf62\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652332 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/05d673e3-59ac-4120-8a1a-90bd25c63823-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652675 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652675 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652389    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-config-volume\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652675 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652423    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05d673e3-59ac-4120-8a1a-90bd25c63823-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.652675 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.652451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.653979 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.653354    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05d673e3-59ac-4120-8a1a-90bd25c63823-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.653979 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.653633    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/05d673e3-59ac-4120-8a1a-90bd25c63823-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.654378 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.654351    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05d673e3-59ac-4120-8a1a-90bd25c63823-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.655347 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.655316    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.655456 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.655426    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.655526 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.655460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/05d673e3-59ac-4120-8a1a-90bd25c63823-tls-assets\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.655526 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.655502    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.655634 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.655618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-web-config\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.655764 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.655742    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.656564 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.656541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/05d673e3-59ac-4120-8a1a-90bd25c63823-config-out\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.656564 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.656558    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-config-volume\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.656696 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.656590    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/05d673e3-59ac-4120-8a1a-90bd25c63823-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.662491 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.662476    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkf62\" (UniqueName: \"kubernetes.io/projected/05d673e3-59ac-4120-8a1a-90bd25c63823-kube-api-access-fkf62\") pod \"alertmanager-main-0\" (UID: \"05d673e3-59ac-4120-8a1a-90bd25c63823\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.785477 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.785383    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 15:51:28.923039 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:28.923008    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 15:51:28.925065 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:51:28.925039    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d673e3_59ac_4120_8a1a_90bd25c63823.slice/crio-ea77a908d7eed7c5d7b0db7470ed89cd65dc6d278b8cca08864b1c1790cf193b WatchSource:0}: Error finding container ea77a908d7eed7c5d7b0db7470ed89cd65dc6d278b8cca08864b1c1790cf193b: Status 404 returned error can't find the container with id ea77a908d7eed7c5d7b0db7470ed89cd65dc6d278b8cca08864b1c1790cf193b
Jul 23 15:51:29.119782 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:29.119694    2562 generic.go:356] "Generic (PLEG): container finished" podID="05d673e3-59ac-4120-8a1a-90bd25c63823" containerID="e5de08647a9d70ce42e2a54d2d275bd394ef889c94b47547e86d91a3f7adb644" exitCode=0
Jul 23 15:51:29.120215 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:29.119782    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d673e3-59ac-4120-8a1a-90bd25c63823","Type":"ContainerDied","Data":"e5de08647a9d70ce42e2a54d2d275bd394ef889c94b47547e86d91a3f7adb644"}
Jul 23 15:51:29.120215 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:29.119875    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d673e3-59ac-4120-8a1a-90bd25c63823","Type":"ContainerStarted","Data":"ea77a908d7eed7c5d7b0db7470ed89cd65dc6d278b8cca08864b1c1790cf193b"}
Jul 23 15:51:29.256576 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:29.256538    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aa2732c-ace1-4462-abde-fb0ad81b0d4d" path="/var/lib/kubelet/pods/0aa2732c-ace1-4462-abde-fb0ad81b0d4d/volumes"
Jul 23 15:51:30.129865 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.129758    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d673e3-59ac-4120-8a1a-90bd25c63823","Type":"ContainerStarted","Data":"ee6560689f1dbc87572f65c9361b919c52ef31139a77c603a32fe625ace55bd6"}
Jul 23 15:51:30.129865 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.129844    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d673e3-59ac-4120-8a1a-90bd25c63823","Type":"ContainerStarted","Data":"958e7969f9d52541cf739286c1c12e4d8f031b74d6a5d4b8960d2e4348b2bfb0"}
Jul 23 15:51:30.130291 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.129871    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d673e3-59ac-4120-8a1a-90bd25c63823","Type":"ContainerStarted","Data":"38043d835d731bebf71e27c352add04ee85c0fa402b5c1a1d45a59acce004454"}
Jul 23 15:51:30.130291 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.129894    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d673e3-59ac-4120-8a1a-90bd25c63823","Type":"ContainerStarted","Data":"8080c9c6516740d07a3f04e567a481fb9e48b7ff1e9f92a91117f927aabb9e74"}
Jul 23 15:51:30.130291 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.129917    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d673e3-59ac-4120-8a1a-90bd25c63823","Type":"ContainerStarted","Data":"539ae5eab03778f65b5205f7f65410f4fa2c99543abbc5aa22941c7061ab98b4"}
Jul 23 15:51:30.130291 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.129939    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d673e3-59ac-4120-8a1a-90bd25c63823","Type":"ContainerStarted","Data":"604219eb21bb4218e54b449ea1b632279d1d07c75b6b156db14db3d7051346c8"}
Jul 23 15:51:30.160931 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.160853    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.13919877 podStartE2EDuration="2.160838032s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.639262ms" isStatefulPod=false podCreationTimestamp="2026-07-23 15:51:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:30.16039596 +0000 UTC m=+153.364763354" watchObservedRunningTime="2026-07-23 15:51:30.160838032 +0000 UTC m=+153.365205427"
Jul 23 15:51:30.834788 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.834758    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-db958c45b-28plx"]
Jul 23 15:51:30.838449 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.838429    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:30.841817 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.841787    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 23 15:51:30.841934 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.841827    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 23 15:51:30.842012 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.841982    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 23 15:51:30.842070 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.842022    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 23 15:51:30.842343 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.842326    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8xsf9\""
Jul 23 15:51:30.843078 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.843065    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 23 15:51:30.850150 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.850128    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 23 15:51:30.859017 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.858994    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-db958c45b-28plx"]
Jul 23 15:51:30.974202 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.974163    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6a8538-6dc9-435b-94eb-eae497220f3c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:30.974202 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.974202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrxhh\" (UniqueName: \"kubernetes.io/projected/af6a8538-6dc9-435b-94eb-eae497220f3c-kube-api-access-qrxhh\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:30.974419 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.974227    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-federate-client-tls\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:30.974419 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.974259    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-secret-telemeter-client\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:30.974419 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.974344    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af6a8538-6dc9-435b-94eb-eae497220f3c-metrics-client-ca\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:30.974543 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.974424    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6a8538-6dc9-435b-94eb-eae497220f3c-serving-certs-ca-bundle\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:30.974543 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.974448    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-telemeter-client-tls\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:30.974543 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:30.974494    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.076054 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.076015    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.076054 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.076059    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6a8538-6dc9-435b-94eb-eae497220f3c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.076312 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.076083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrxhh\" (UniqueName: \"kubernetes.io/projected/af6a8538-6dc9-435b-94eb-eae497220f3c-kube-api-access-qrxhh\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.076312 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.076201    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-federate-client-tls\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.076312 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.076246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-secret-telemeter-client\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.076312 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.076289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af6a8538-6dc9-435b-94eb-eae497220f3c-metrics-client-ca\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.076559 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.076370    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6a8538-6dc9-435b-94eb-eae497220f3c-serving-certs-ca-bundle\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.076559 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.076395    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-telemeter-client-tls\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.077142 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.077108    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6a8538-6dc9-435b-94eb-eae497220f3c-serving-certs-ca-bundle\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.077283 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.077108    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af6a8538-6dc9-435b-94eb-eae497220f3c-metrics-client-ca\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.077283 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.077253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6a8538-6dc9-435b-94eb-eae497220f3c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.078662 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.078630    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.078896 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.078866    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-telemeter-client-tls\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.078966 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.078916    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-federate-client-tls\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.079001 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.078970    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/af6a8538-6dc9-435b-94eb-eae497220f3c-secret-telemeter-client\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.084718 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.084695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrxhh\" (UniqueName: \"kubernetes.io/projected/af6a8538-6dc9-435b-94eb-eae497220f3c-kube-api-access-qrxhh\") pod \"telemeter-client-db958c45b-28plx\" (UID: \"af6a8538-6dc9-435b-94eb-eae497220f3c\") " pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.149172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.149087    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-db958c45b-28plx"
Jul 23 15:51:31.187297 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.187260    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 15:51:31.187711 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.187678    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="prometheus" containerID="cri-o://07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561" gracePeriod=600
Jul 23 15:51:31.187853 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.187713    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy" containerID="cri-o://7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533" gracePeriod=600
Jul 23 15:51:31.187853 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.187786    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="thanos-sidecar" containerID="cri-o://f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4" gracePeriod=600
Jul 23 15:51:31.187966 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.187886    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="config-reloader" containerID="cri-o://648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade" gracePeriod=600
Jul 23 15:51:31.187966 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.187911    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy-thanos" containerID="cri-o://83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6" gracePeriod=600
Jul 23 15:51:31.187966 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.187929    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy-web" containerID="cri-o://3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5" gracePeriod=600
Jul 23 15:51:31.318576 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.318551    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-db958c45b-28plx"]
Jul 23 15:51:31.320112 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:51:31.320090    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6a8538_6dc9_435b_94eb_eae497220f3c.slice/crio-91177629c31ec9df022b170180af6292beea1526f311ea496f2ebe761e9d431a WatchSource:0}: Error finding container 91177629c31ec9df022b170180af6292beea1526f311ea496f2ebe761e9d431a: Status 404 returned error can't find the container with id 91177629c31ec9df022b170180af6292beea1526f311ea496f2ebe761e9d431a
Jul 23 15:51:31.467389 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.467361    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:31.584042 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584006    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-0\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584058    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-kube-rbac-proxy\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584116    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-2\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584148    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-thanos-prometheus-http-client-file\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584180    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-metrics-client-certs\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584217 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584208    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-trusted-ca-bundle\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584478 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584240    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpjg5\" (UniqueName: \"kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-kube-api-access-qpjg5\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584478 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584268    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-config\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584478 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584295    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-config-out\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584478 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584325    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-serving-certs-ca-bundle\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584478 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584349    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-tls-assets\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584478 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584386    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584478 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584414    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584478 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584437    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-db\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584478 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584462    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:51:31.584923 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584471    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-1\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584923 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584536    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-metrics-client-ca\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584923 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584589    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-web-config\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584923 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584621    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:51:31.584923 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584636    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-tls\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584923 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584661    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-grpc-tls\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584923 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584688    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-kubelet-serving-ca-bundle\") pod \"bf786fed-02bb-473b-873a-b06582337668\" (UID: \"bf786fed-02bb-473b-873a-b06582337668\") "
Jul 23 15:51:31.584923 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584736    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:51:31.584923 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.584897    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:51:31.585391 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.585031    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.585391 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.585051    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.585391 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.585071    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.585391 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.585086    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.585391 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.585155    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:51:31.585624 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.585397    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:51:31.585837 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.585793    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:51:31.587182 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.586882    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:31.587182 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.586929    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:31.587182 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.586963    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:31.587491 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.587183    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 15:51:31.587673 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.587638    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-kube-api-access-qpjg5" (OuterVolumeSpecName: "kube-api-access-qpjg5") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "kube-api-access-qpjg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 15:51:31.588254 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.588191    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:31.588501 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.588469    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:31.588587 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.588486    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 15:51:31.589225 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.589204    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:31.589225 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.589212    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-config-out" (OuterVolumeSpecName: "config-out") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 15:51:31.589495 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.589477    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-config" (OuterVolumeSpecName: "config") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:31.589601 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.589586    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:31.599080 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.599059    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-web-config" (OuterVolumeSpecName: "web-config") pod "bf786fed-02bb-473b-873a-b06582337668" (UID: "bf786fed-02bb-473b-873a-b06582337668"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:51:31.685943 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.685908    2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.685943 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.685937    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.685943 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.685947    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.685956    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.685965    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.685976    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-db\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.685987    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-metrics-client-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.685995    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.686003    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.686013    2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-grpc-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.686022    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.686030    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bf786fed-02bb-473b-873a-b06582337668-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.686039    2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-kube-rbac-proxy\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.686047    2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.686056    2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bf786fed-02bb-473b-873a-b06582337668-secret-metrics-client-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:31.686172 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:31.686063    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpjg5\" (UniqueName: \"kubernetes.io/projected/bf786fed-02bb-473b-873a-b06582337668-kube-api-access-qpjg5\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:51:32.142446 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.142410    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-db958c45b-28plx" event={"ID":"af6a8538-6dc9-435b-94eb-eae497220f3c","Type":"ContainerStarted","Data":"91177629c31ec9df022b170180af6292beea1526f311ea496f2ebe761e9d431a"}
Jul 23 15:51:32.146325 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146292    2562 generic.go:356] "Generic (PLEG): container finished" podID="bf786fed-02bb-473b-873a-b06582337668" containerID="83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6" exitCode=0
Jul 23 15:51:32.146325 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146320    2562 generic.go:356] "Generic (PLEG): container finished" podID="bf786fed-02bb-473b-873a-b06582337668" containerID="7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533" exitCode=0
Jul 23 15:51:32.146503 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146331    2562 generic.go:356] "Generic (PLEG): container finished" podID="bf786fed-02bb-473b-873a-b06582337668" containerID="3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5" exitCode=0
Jul 23 15:51:32.146503 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146340    2562 generic.go:356] "Generic (PLEG): container finished" podID="bf786fed-02bb-473b-873a-b06582337668" containerID="f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4" exitCode=0
Jul 23 15:51:32.146503 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146349    2562 generic.go:356] "Generic (PLEG): container finished" podID="bf786fed-02bb-473b-873a-b06582337668" containerID="648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade" exitCode=0
Jul 23 15:51:32.146503 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146359    2562 generic.go:356] "Generic (PLEG): container finished" podID="bf786fed-02bb-473b-873a-b06582337668" containerID="07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561" exitCode=0
Jul 23 15:51:32.146503 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146369    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerDied","Data":"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"}
Jul 23 15:51:32.146503 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146445    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerDied","Data":"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"}
Jul 23 15:51:32.146503 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146380    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.146503 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146489    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerDied","Data":"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"}
Jul 23 15:51:32.146837 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146531    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerDied","Data":"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"}
Jul 23 15:51:32.146837 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146533    2562 scope.go:117] "RemoveContainer" containerID="83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"
Jul 23 15:51:32.146837 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146570    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerDied","Data":"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"}
Jul 23 15:51:32.146837 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146611    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerDied","Data":"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"}
Jul 23 15:51:32.146837 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.146653    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf786fed-02bb-473b-873a-b06582337668","Type":"ContainerDied","Data":"9eaced9980065496cd1ddb63e94e201d234f60bea61ad1a5d40056c5d0881e87"}
Jul 23 15:51:32.157292 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.157200    2562 scope.go:117] "RemoveContainer" containerID="7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"
Jul 23 15:51:32.165692 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.165674    2562 scope.go:117] "RemoveContainer" containerID="3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"
Jul 23 15:51:32.175168 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.175149    2562 scope.go:117] "RemoveContainer" containerID="f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"
Jul 23 15:51:32.175252 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.175216    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 15:51:32.179425 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.179400    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 15:51:32.183641 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.183620    2562 scope.go:117] "RemoveContainer" containerID="648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"
Jul 23 15:51:32.192455 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.192436    2562 scope.go:117] "RemoveContainer" containerID="07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"
Jul 23 15:51:32.203291 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.203268    2562 scope.go:117] "RemoveContainer" containerID="2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"
Jul 23 15:51:32.204313 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204286    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 15:51:32.204753 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204733    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy"
Jul 23 15:51:32.204887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204755    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy"
Jul 23 15:51:32.204887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204772    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy-web"
Jul 23 15:51:32.204887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204780    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy-web"
Jul 23 15:51:32.204887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204792    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy-thanos"
Jul 23 15:51:32.204887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204814    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy-thanos"
Jul 23 15:51:32.204887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204835    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="config-reloader"
Jul 23 15:51:32.204887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204846    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="config-reloader"
Jul 23 15:51:32.204887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204863    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="init-config-reloader"
Jul 23 15:51:32.204887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204870    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="init-config-reloader"
Jul 23 15:51:32.204887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204881    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="prometheus"
Jul 23 15:51:32.204887 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204889    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="prometheus"
Jul 23 15:51:32.205589 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204898    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="thanos-sidecar"
Jul 23 15:51:32.205589 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.204906    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="thanos-sidecar"
Jul 23 15:51:32.205589 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.205031    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="config-reloader"
Jul 23 15:51:32.205589 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.205049    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="prometheus"
Jul 23 15:51:32.205589 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.205057    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy-web"
Jul 23 15:51:32.205589 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.205068    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy"
Jul 23 15:51:32.205589 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.205078    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="thanos-sidecar"
Jul 23 15:51:32.205589 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.205088    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf786fed-02bb-473b-873a-b06582337668" containerName="kube-rbac-proxy-thanos"
Jul 23 15:51:32.213321 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.213299    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.213990 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.213693    2562 scope.go:117] "RemoveContainer" containerID="83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"
Jul 23 15:51:32.214147 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:32.214121    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": container with ID starting with 83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6 not found: ID does not exist" containerID="83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"
Jul 23 15:51:32.214215 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.214167    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"} err="failed to get container status \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": rpc error: code = NotFound desc = could not find container \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": container with ID starting with 83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6 not found: ID does not exist"
Jul 23 15:51:32.214215 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.214194    2562 scope.go:117] "RemoveContainer" containerID="7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"
Jul 23 15:51:32.214497 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:32.214472    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": container with ID starting with 7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533 not found: ID does not exist" containerID="7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"
Jul 23 15:51:32.214548 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.214506    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"} err="failed to get container status \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": rpc error: code = NotFound desc = could not find container \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": container with ID starting with 7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533 not found: ID does not exist"
Jul 23 15:51:32.214548 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.214522    2562 scope.go:117] "RemoveContainer" containerID="3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"
Jul 23 15:51:32.214908 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:32.214792    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": container with ID starting with 3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5 not found: ID does not exist" containerID="3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"
Jul 23 15:51:32.214908 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.214866    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"} err="failed to get container status \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": rpc error: code = NotFound desc = could not find container \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": container with ID starting with 3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5 not found: ID does not exist"
Jul 23 15:51:32.214908 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.214887    2562 scope.go:117] "RemoveContainer" containerID="f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"
Jul 23 15:51:32.215289 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:32.215185    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": container with ID starting with f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4 not found: ID does not exist" containerID="f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"
Jul 23 15:51:32.215289 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.215216    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"} err="failed to get container status \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": rpc error: code = NotFound desc = could not find container \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": container with ID starting with f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4 not found: ID does not exist"
Jul 23 15:51:32.215289 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.215238    2562 scope.go:117] "RemoveContainer" containerID="648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"
Jul 23 15:51:32.215589 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:32.215564    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": container with ID starting with 648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade not found: ID does not exist" containerID="648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"
Jul 23 15:51:32.215644 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.215597    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"} err="failed to get container status \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": rpc error: code = NotFound desc = could not find container \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": container with ID starting with 648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade not found: ID does not exist"
Jul 23 15:51:32.215644 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.215618    2562 scope.go:117] "RemoveContainer" containerID="07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"
Jul 23 15:51:32.215981 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:32.215959    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": container with ID starting with 07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561 not found: ID does not exist" containerID="07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"
Jul 23 15:51:32.216066 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.215988    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"} err="failed to get container status \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": rpc error: code = NotFound desc = could not find container \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": container with ID starting with 07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561 not found: ID does not exist"
Jul 23 15:51:32.216066 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216008    2562 scope.go:117] "RemoveContainer" containerID="2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"
Jul 23 15:51:32.216349 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:51:32.216322    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": container with ID starting with 2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e not found: ID does not exist" containerID="2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"
Jul 23 15:51:32.216445 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216354    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"} err="failed to get container status \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": rpc error: code = NotFound desc = could not find container \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": container with ID starting with 2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e not found: ID does not exist"
Jul 23 15:51:32.216445 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216382    2562 scope.go:117] "RemoveContainer" containerID="83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"
Jul 23 15:51:32.216530 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216468    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 23 15:51:32.216568 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216531    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 23 15:51:32.216600 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216577    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 23 15:51:32.216632 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216469    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 23 15:51:32.216667 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216630    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 23 15:51:32.216792 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216776    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 23 15:51:32.216879 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216846    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 23 15:51:32.216879 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216856    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 23 15:51:32.216986 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216933    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9v93cbs5vmt17\""
Jul 23 15:51:32.216986 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.216852    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 23 15:51:32.217212 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.217181    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-67zb2\""
Jul 23 15:51:32.217212 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.217202    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 23 15:51:32.217369 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.217187    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 23 15:51:32.217369 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.217181    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 23 15:51:32.217369 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.217287    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"} err="failed to get container status \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": rpc error: code = NotFound desc = could not find container \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": container with ID starting with 83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6 not found: ID does not exist"
Jul 23 15:51:32.217369 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.217315    2562 scope.go:117] "RemoveContainer" containerID="7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"
Jul 23 15:51:32.217666 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.217639    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"} err="failed to get container status \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": rpc error: code = NotFound desc = could not find container \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": container with ID starting with 7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533 not found: ID does not exist"
Jul 23 15:51:32.217753 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.217669    2562 scope.go:117] "RemoveContainer" containerID="3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"
Jul 23 15:51:32.218735 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.218198    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"} err="failed to get container status \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": rpc error: code = NotFound desc = could not find container \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": container with ID starting with 3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5 not found: ID does not exist"
Jul 23 15:51:32.218735 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.218228    2562 scope.go:117] "RemoveContainer" containerID="f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"
Jul 23 15:51:32.218735 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.218636    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"} err="failed to get container status \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": rpc error: code = NotFound desc = could not find container \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": container with ID starting with f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4 not found: ID does not exist"
Jul 23 15:51:32.218735 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.218659    2562 scope.go:117] "RemoveContainer" containerID="648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"
Jul 23 15:51:32.219865 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.219795    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"} err="failed to get container status \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": rpc error: code = NotFound desc = could not find container \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": container with ID starting with 648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade not found: ID does not exist"
Jul 23 15:51:32.219865 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.219840    2562 scope.go:117] "RemoveContainer" containerID="07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"
Jul 23 15:51:32.220042 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.219994    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 23 15:51:32.220101 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.220073    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"} err="failed to get container status \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": rpc error: code = NotFound desc = could not find container \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": container with ID starting with 07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561 not found: ID does not exist"
Jul 23 15:51:32.220166 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.220104    2562 scope.go:117] "RemoveContainer" containerID="2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"
Jul 23 15:51:32.220407 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.220374    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"} err="failed to get container status \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": rpc error: code = NotFound desc = could not find container \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": container with ID starting with 2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e not found: ID does not exist"
Jul 23 15:51:32.220407 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.220403    2562 scope.go:117] "RemoveContainer" containerID="83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"
Jul 23 15:51:32.220756 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.220685    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"} err="failed to get container status \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": rpc error: code = NotFound desc = could not find container \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": container with ID starting with 83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6 not found: ID does not exist"
Jul 23 15:51:32.220756 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.220711    2562 scope.go:117] "RemoveContainer" containerID="7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"
Jul 23 15:51:32.221243 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.221157    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"} err="failed to get container status \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": rpc error: code = NotFound desc = could not find container \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": container with ID starting with 7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533 not found: ID does not exist"
Jul 23 15:51:32.221243 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.221182    2562 scope.go:117] "RemoveContainer" containerID="3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"
Jul 23 15:51:32.222171 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.222094    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"} err="failed to get container status \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": rpc error: code = NotFound desc = could not find container \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": container with ID starting with 3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5 not found: ID does not exist"
Jul 23 15:51:32.222171 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.222119    2562 scope.go:117] "RemoveContainer" containerID="f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"
Jul 23 15:51:32.222984 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.222886    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"} err="failed to get container status \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": rpc error: code = NotFound desc = could not find container \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": container with ID starting with f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4 not found: ID does not exist"
Jul 23 15:51:32.222984 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.222912    2562 scope.go:117] "RemoveContainer" containerID="648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"
Jul 23 15:51:32.223444 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.223422    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 23 15:51:32.224664 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.224578    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 15:51:32.224926 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.224853    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"} err="failed to get container status \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": rpc error: code = NotFound desc = could not find container \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": container with ID starting with 648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade not found: ID does not exist"
Jul 23 15:51:32.224926 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.224881    2562 scope.go:117] "RemoveContainer" containerID="07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"
Jul 23 15:51:32.225855 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.225826    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"} err="failed to get container status \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": rpc error: code = NotFound desc = could not find container \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": container with ID starting with 07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561 not found: ID does not exist"
Jul 23 15:51:32.225855 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.225854    2562 scope.go:117] "RemoveContainer" containerID="2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"
Jul 23 15:51:32.226205 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.226104    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"} err="failed to get container status \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": rpc error: code = NotFound desc = could not find container \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": container with ID starting with 2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e not found: ID does not exist"
Jul 23 15:51:32.226205 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.226130    2562 scope.go:117] "RemoveContainer" containerID="83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"
Jul 23 15:51:32.226685 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.226641    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"} err="failed to get container status \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": rpc error: code = NotFound desc = could not find container \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": container with ID starting with 83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6 not found: ID does not exist"
Jul 23 15:51:32.226685 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.226669    2562 scope.go:117] "RemoveContainer" containerID="7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"
Jul 23 15:51:32.227017 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.226974    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"} err="failed to get container status \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": rpc error: code = NotFound desc = could not find container \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": container with ID starting with 7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533 not found: ID does not exist"
Jul 23 15:51:32.227017 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.227011    2562 scope.go:117] "RemoveContainer" containerID="3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"
Jul 23 15:51:32.227371 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.227268    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"} err="failed to get container status \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": rpc error: code = NotFound desc = could not find container \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": container with ID starting with 3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5 not found: ID does not exist"
Jul 23 15:51:32.227371 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.227294    2562 scope.go:117] "RemoveContainer" containerID="f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"
Jul 23 15:51:32.227612 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.227583    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"} err="failed to get container status \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": rpc error: code = NotFound desc = could not find container \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": container with ID starting with f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4 not found: ID does not exist"
Jul 23 15:51:32.227612 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.227607    2562 scope.go:117] "RemoveContainer" containerID="648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"
Jul 23 15:51:32.227894 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.227873    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"} err="failed to get container status \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": rpc error: code = NotFound desc = could not find container \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": container with ID starting with 648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade not found: ID does not exist"
Jul 23 15:51:32.227894 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.227892    2562 scope.go:117] "RemoveContainer" containerID="07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"
Jul 23 15:51:32.228162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.228141    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"} err="failed to get container status \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": rpc error: code = NotFound desc = could not find container \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": container with ID starting with 07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561 not found: ID does not exist"
Jul 23 15:51:32.228221 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.228164    2562 scope.go:117] "RemoveContainer" containerID="2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"
Jul 23 15:51:32.228506 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.228483    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"} err="failed to get container status \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": rpc error: code = NotFound desc = could not find container \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": container with ID starting with 2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e not found: ID does not exist"
Jul 23 15:51:32.228506 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.228506    2562 scope.go:117] "RemoveContainer" containerID="83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"
Jul 23 15:51:32.228859 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.228825    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"} err="failed to get container status \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": rpc error: code = NotFound desc = could not find container \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": container with ID starting with 83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6 not found: ID does not exist"
Jul 23 15:51:32.228859 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.228849    2562 scope.go:117] "RemoveContainer" containerID="7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"
Jul 23 15:51:32.229095 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.229074    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"} err="failed to get container status \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": rpc error: code = NotFound desc = could not find container \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": container with ID starting with 7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533 not found: ID does not exist"
Jul 23 15:51:32.229095 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.229094    2562 scope.go:117] "RemoveContainer" containerID="3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"
Jul 23 15:51:32.229375 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.229356    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"} err="failed to get container status \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": rpc error: code = NotFound desc = could not find container \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": container with ID starting with 3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5 not found: ID does not exist"
Jul 23 15:51:32.229451 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.229376    2562 scope.go:117] "RemoveContainer" containerID="f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"
Jul 23 15:51:32.229672 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.229648    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"} err="failed to get container status \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": rpc error: code = NotFound desc = could not find container \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": container with ID starting with f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4 not found: ID does not exist"
Jul 23 15:51:32.229739 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.229675    2562 scope.go:117] "RemoveContainer" containerID="648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"
Jul 23 15:51:32.229965 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.229945    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"} err="failed to get container status \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": rpc error: code = NotFound desc = could not find container \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": container with ID starting with 648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade not found: ID does not exist"
Jul 23 15:51:32.230064 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.229967    2562 scope.go:117] "RemoveContainer" containerID="07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"
Jul 23 15:51:32.230347 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.230321    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"} err="failed to get container status \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": rpc error: code = NotFound desc = could not find container \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": container with ID starting with 07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561 not found: ID does not exist"
Jul 23 15:51:32.230484 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.230347    2562 scope.go:117] "RemoveContainer" containerID="2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"
Jul 23 15:51:32.230901 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.230872    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"} err="failed to get container status \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": rpc error: code = NotFound desc = could not find container \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": container with ID starting with 2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e not found: ID does not exist"
Jul 23 15:51:32.231023 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.230917    2562 scope.go:117] "RemoveContainer" containerID="83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"
Jul 23 15:51:32.231619 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.231445    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6"} err="failed to get container status \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": rpc error: code = NotFound desc = could not find container \"83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6\": container with ID starting with 83d2abbb41e5767bfbd6b5c1e6236b377216d32635f72162be30721485c7a8b6 not found: ID does not exist"
Jul 23 15:51:32.231619 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.231470    2562 scope.go:117] "RemoveContainer" containerID="7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"
Jul 23 15:51:32.232013 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.231992    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533"} err="failed to get container status \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": rpc error: code = NotFound desc = could not find container \"7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533\": container with ID starting with 7a6be56b0e66ce876c36e423ff1320599b1967cc0bf6621058a217a3c4076533 not found: ID does not exist"
Jul 23 15:51:32.232113 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.232015    2562 scope.go:117] "RemoveContainer" containerID="3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"
Jul 23 15:51:32.232325 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.232306    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5"} err="failed to get container status \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": rpc error: code = NotFound desc = could not find container \"3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5\": container with ID starting with 3ff133ec62d6f905c4287daf10afd921ca5ea3e49b36a7c568a4da98926cadd5 not found: ID does not exist"
Jul 23 15:51:32.232427 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.232327    2562 scope.go:117] "RemoveContainer" containerID="f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"
Jul 23 15:51:32.232596 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.232579    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4"} err="failed to get container status \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": rpc error: code = NotFound desc = could not find container \"f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4\": container with ID starting with f685e915a8dd82da78dfd686f7a87f4f1557cf7677da7f1eb415327f63f96ad4 not found: ID does not exist"
Jul 23 15:51:32.232676 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.232598    2562 scope.go:117] "RemoveContainer" containerID="648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"
Jul 23 15:51:32.232874 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.232850    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade"} err="failed to get container status \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": rpc error: code = NotFound desc = could not find container \"648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade\": container with ID starting with 648ab7a9b6627b510dc055254ca2a5b32309f45b5988158152fbec7f219c9ade not found: ID does not exist"
Jul 23 15:51:32.232874 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.232874    2562 scope.go:117] "RemoveContainer" containerID="07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"
Jul 23 15:51:32.233114 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.233099    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561"} err="failed to get container status \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": rpc error: code = NotFound desc = could not find container \"07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561\": container with ID starting with 07d36a790acbd97e43bedfca9fc628975f5b671bbd34893d8d0a7510515df561 not found: ID does not exist"
Jul 23 15:51:32.233114 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.233114    2562 scope.go:117] "RemoveContainer" containerID="2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"
Jul 23 15:51:32.233349 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.233325    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e"} err="failed to get container status \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": rpc error: code = NotFound desc = could not find container \"2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e\": container with ID starting with 2eea6ffded3c8beeb4a6f7e43ae7eb7261a526735627686ce33f7f8a4bc5a87e not found: ID does not exist"
Jul 23 15:51:32.292274 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292241    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292274 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292534 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292312    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292534 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292421    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292534 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292456    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292534 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0600ef0a-4537-45d5-a61a-a3bef7fc6323-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292534 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292534    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292783 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292563    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292783 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292599    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-web-config\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292783 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292783 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292690    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-config\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292783 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292725    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.292783 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292751    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.293070 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292782    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.293070 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.293070 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292907    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.293070 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ff2p\" (UniqueName: \"kubernetes.io/projected/0600ef0a-4537-45d5-a61a-a3bef7fc6323-kube-api-access-8ff2p\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.293070 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.292991    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.293274 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.293072    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.293274 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.293114    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0600ef0a-4537-45d5-a61a-a3bef7fc6323-config-out\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.393598 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.393512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.393598 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.393554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ff2p\" (UniqueName: \"kubernetes.io/projected/0600ef0a-4537-45d5-a61a-a3bef7fc6323-kube-api-access-8ff2p\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.393858 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.393834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.393941 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.393925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394000 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.393986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0600ef0a-4537-45d5-a61a-a3bef7fc6323-config-out\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394050 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.394030    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394104 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.394060    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394104 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.394098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394200 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.394137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394200 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.394166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394299 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.394205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0600ef0a-4537-45d5-a61a-a3bef7fc6323-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394299 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.394247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394395 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.394300    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394395 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.394330    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-web-config\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394490 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.394408    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.394490 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.394437    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-config\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.396939 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.395478    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.396939 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.396258    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.396939 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.396312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.396939 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.396351    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.396939 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.396377    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.396939 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.396447    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.397882 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.397594    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.397882 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.397621    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0600ef0a-4537-45d5-a61a-a3bef7fc6323-config-out\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.397882 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.397796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.398375 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.398230    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.398375 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.398265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.399388 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.399017    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.399388 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.399245    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.399872 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.399851    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.400552 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.400028    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.401249 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.401112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-web-config\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.401970 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.401912    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.401970 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.401943    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.402339 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.402280    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0600ef0a-4537-45d5-a61a-a3bef7fc6323-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.404250 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.402579    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-config\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.404250 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.402901    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.404250 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.402966    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0600ef0a-4537-45d5-a61a-a3bef7fc6323-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.405604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.404675    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ff2p\" (UniqueName: \"kubernetes.io/projected/0600ef0a-4537-45d5-a61a-a3bef7fc6323-kube-api-access-8ff2p\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.405604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.404679    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0600ef0a-4537-45d5-a61a-a3bef7fc6323-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0600ef0a-4537-45d5-a61a-a3bef7fc6323\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.526063 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.526027    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:32.675484 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:32.675458    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 15:51:32.677367 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:51:32.677331    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0600ef0a_4537_45d5_a61a_a3bef7fc6323.slice/crio-736bf6032d1da8e934549db3325ebe0aac437bf64c7a7821842c431110b4d397 WatchSource:0}: Error finding container 736bf6032d1da8e934549db3325ebe0aac437bf64c7a7821842c431110b4d397: Status 404 returned error can't find the container with id 736bf6032d1da8e934549db3325ebe0aac437bf64c7a7821842c431110b4d397
Jul 23 15:51:33.152323 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:33.152220    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0600ef0a-4537-45d5-a61a-a3bef7fc6323","Type":"ContainerStarted","Data":"19bc3896e3d277c6b2bd3764c7c74f353f23297342d73655ec0e01936d79b97c"}
Jul 23 15:51:33.152323 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:33.152313    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0600ef0a-4537-45d5-a61a-a3bef7fc6323","Type":"ContainerStarted","Data":"736bf6032d1da8e934549db3325ebe0aac437bf64c7a7821842c431110b4d397"}
Jul 23 15:51:33.254063 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:33.254036    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf786fed-02bb-473b-873a-b06582337668" path="/var/lib/kubelet/pods/bf786fed-02bb-473b-873a-b06582337668/volumes"
Jul 23 15:51:34.160264 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:34.160230    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-db958c45b-28plx" event={"ID":"af6a8538-6dc9-435b-94eb-eae497220f3c","Type":"ContainerStarted","Data":"e107ad9c205bfb175864a1b4179a13675bd4507c2104b9817d542797a17ed7a1"}
Jul 23 15:51:34.160486 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:34.160294    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-db958c45b-28plx" event={"ID":"af6a8538-6dc9-435b-94eb-eae497220f3c","Type":"ContainerStarted","Data":"83769cba2db33b913136d89be46e1e16743bbbfe8138fb788cf0570e3becda84"}
Jul 23 15:51:34.160486 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:34.160320    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-db958c45b-28plx" event={"ID":"af6a8538-6dc9-435b-94eb-eae497220f3c","Type":"ContainerStarted","Data":"c205a1dc3c8a94604de0d12a9dfeaa63de7ccaeb4766df69753419a6475f14fa"}
Jul 23 15:51:34.161754 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:34.161731    2562 generic.go:356] "Generic (PLEG): container finished" podID="0600ef0a-4537-45d5-a61a-a3bef7fc6323" containerID="19bc3896e3d277c6b2bd3764c7c74f353f23297342d73655ec0e01936d79b97c" exitCode=0
Jul 23 15:51:34.161900 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:34.161780    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0600ef0a-4537-45d5-a61a-a3bef7fc6323","Type":"ContainerDied","Data":"19bc3896e3d277c6b2bd3764c7c74f353f23297342d73655ec0e01936d79b97c"}
Jul 23 15:51:34.184528 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:34.184477    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-db958c45b-28plx" podStartSLOduration=2.298408636 podStartE2EDuration="4.184458313s" totalImagesPullingTime="1.886049677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:34.183055892 +0000 UTC m=+157.387423309" watchObservedRunningTime="2026-07-23 15:51:34.184458313 +0000 UTC m=+157.388825746"
Jul 23 15:51:34.975588 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:34.975547    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-747488c4bb-wp7dl"]
Jul 23 15:51:34.978930 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:34.978903    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:34.993225 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:34.993195    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-747488c4bb-wp7dl"]
Jul 23 15:51:35.122365 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.122316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-serving-cert\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.122552 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.122378    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-oauth-config\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.122552 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.122405    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-service-ca\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.122552 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.122425    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-config\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.122552 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.122439    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-trusted-ca-bundle\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.122793 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.122564    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-oauth-serving-cert\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.122793 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.122595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzg6n\" (UniqueName: \"kubernetes.io/projected/2d5c430c-3f14-4107-a861-78fd445bcb4f-kube-api-access-wzg6n\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.169748 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.169711    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0600ef0a-4537-45d5-a61a-a3bef7fc6323","Type":"ContainerStarted","Data":"e635e081c50fdb8253c2512877541d2fad52e80358f6b988e45d447b094dbd2a"}
Jul 23 15:51:35.169960 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.169778    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0600ef0a-4537-45d5-a61a-a3bef7fc6323","Type":"ContainerStarted","Data":"fe8f2b3645a58eeb4f746968cb7dd13f364e07325fe6db821ca9771ae9fab044"}
Jul 23 15:51:35.169960 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.169823    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0600ef0a-4537-45d5-a61a-a3bef7fc6323","Type":"ContainerStarted","Data":"9bf1a7a73ee29aa53f34d7d293975174ce220833a0602d878668a86fb1605d74"}
Jul 23 15:51:35.169960 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.169857    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0600ef0a-4537-45d5-a61a-a3bef7fc6323","Type":"ContainerStarted","Data":"25eebc5a16181f8e9e2e2a30afeedacad5d2d27077849b098a1db388733f9f2b"}
Jul 23 15:51:35.169960 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.169882    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0600ef0a-4537-45d5-a61a-a3bef7fc6323","Type":"ContainerStarted","Data":"83dfba142fd4fc951fd91d056666f0ae7b937142a4fdd72ea16b352b2ef331b6"}
Jul 23 15:51:35.169960 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.169904    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0600ef0a-4537-45d5-a61a-a3bef7fc6323","Type":"ContainerStarted","Data":"1ae5cf4343b615835f2ebe7dbef014c56af4536ecf987c88ebad09065cd3231f"}
Jul 23 15:51:35.203412 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.203237    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.183895829 podStartE2EDuration="3.203213581s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.317752ms" isStatefulPod=false podCreationTimestamp="2026-07-23 15:51:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:35.202538153 +0000 UTC m=+158.406905548" watchObservedRunningTime="2026-07-23 15:51:35.203213581 +0000 UTC m=+158.407580976"
Jul 23 15:51:35.223379 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.223338    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-oauth-serving-cert\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.223379 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.223386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzg6n\" (UniqueName: \"kubernetes.io/projected/2d5c430c-3f14-4107-a861-78fd445bcb4f-kube-api-access-wzg6n\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.223590 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.223454    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-serving-cert\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.223590 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.223520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-oauth-config\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.223590 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.223547    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-service-ca\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.224007 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.223978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-config\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.224134 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.224030    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-trusted-ca-bundle\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.224258 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.224229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-oauth-serving-cert\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.224339 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.224278    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-service-ca\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.224455 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.224437    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-config\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.224778 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.224762    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-trusted-ca-bundle\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.226187 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.226127    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-oauth-config\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.226394 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.226375    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-serving-cert\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.232577 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.232553    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzg6n\" (UniqueName: \"kubernetes.io/projected/2d5c430c-3f14-4107-a861-78fd445bcb4f-kube-api-access-wzg6n\") pod \"console-747488c4bb-wp7dl\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") " pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.290653 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.290614    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:35.435858 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:35.435830    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-747488c4bb-wp7dl"]
Jul 23 15:51:35.438071 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:51:35.438038    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d5c430c_3f14_4107_a861_78fd445bcb4f.slice/crio-9af2e74234051701e8b072336e84d5f81afadcc891d57483a779bb7fa1d77df2 WatchSource:0}: Error finding container 9af2e74234051701e8b072336e84d5f81afadcc891d57483a779bb7fa1d77df2: Status 404 returned error can't find the container with id 9af2e74234051701e8b072336e84d5f81afadcc891d57483a779bb7fa1d77df2
Jul 23 15:51:36.177300 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:36.177257    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747488c4bb-wp7dl" event={"ID":"2d5c430c-3f14-4107-a861-78fd445bcb4f","Type":"ContainerStarted","Data":"93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406"}
Jul 23 15:51:36.177725 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:36.177316    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747488c4bb-wp7dl" event={"ID":"2d5c430c-3f14-4107-a861-78fd445bcb4f","Type":"ContainerStarted","Data":"9af2e74234051701e8b072336e84d5f81afadcc891d57483a779bb7fa1d77df2"}
Jul 23 15:51:36.195559 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:36.195510    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-747488c4bb-wp7dl" podStartSLOduration=2.195493912 podStartE2EDuration="2.195493912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:51:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:36.194044438 +0000 UTC m=+159.398411861" watchObservedRunningTime="2026-07-23 15:51:36.195493912 +0000 UTC m=+159.399861305"
Jul 23 15:51:37.526154 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:37.526119    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:51:45.291474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:45.291434    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:45.291957 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:45.291495    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:45.297192 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:45.297167    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:46.227340 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:46.227311    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:51:46.276387 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:51:46.276360    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d788c457-gww6n"]
Jul 23 15:52:11.297292 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.297224    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d788c457-gww6n" podUID="d8680750-4437-46fe-bc00-4c65413611ce" containerName="console" containerID="cri-o://f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd" gracePeriod=15
Jul 23 15:52:11.546389 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.546362    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:52:11.658010 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.657927    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-serving-cert\") pod \"d8680750-4437-46fe-bc00-4c65413611ce\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") "
Jul 23 15:52:11.658010 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.657976    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47qd5\" (UniqueName: \"kubernetes.io/projected/d8680750-4437-46fe-bc00-4c65413611ce-kube-api-access-47qd5\") pod \"d8680750-4437-46fe-bc00-4c65413611ce\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") "
Jul 23 15:52:11.658223 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.658034    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-oauth-serving-cert\") pod \"d8680750-4437-46fe-bc00-4c65413611ce\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") "
Jul 23 15:52:11.658223 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.658055    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-trusted-ca-bundle\") pod \"d8680750-4437-46fe-bc00-4c65413611ce\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") "
Jul 23 15:52:11.658223 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.658081    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-console-config\") pod \"d8680750-4437-46fe-bc00-4c65413611ce\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") "
Jul 23 15:52:11.658223 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.658098    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-service-ca\") pod \"d8680750-4437-46fe-bc00-4c65413611ce\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") "
Jul 23 15:52:11.658223 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.658184    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-oauth-config\") pod \"d8680750-4437-46fe-bc00-4c65413611ce\" (UID: \"d8680750-4437-46fe-bc00-4c65413611ce\") "
Jul 23 15:52:11.658584 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.658528    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d8680750-4437-46fe-bc00-4c65413611ce" (UID: "d8680750-4437-46fe-bc00-4c65413611ce"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:52:11.658584 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.658541    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-console-config" (OuterVolumeSpecName: "console-config") pod "d8680750-4437-46fe-bc00-4c65413611ce" (UID: "d8680750-4437-46fe-bc00-4c65413611ce"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:52:11.658584 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.658551    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d8680750-4437-46fe-bc00-4c65413611ce" (UID: "d8680750-4437-46fe-bc00-4c65413611ce"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:52:11.658584 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.658537    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-service-ca" (OuterVolumeSpecName: "service-ca") pod "d8680750-4437-46fe-bc00-4c65413611ce" (UID: "d8680750-4437-46fe-bc00-4c65413611ce"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:52:11.660381 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.660354    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d8680750-4437-46fe-bc00-4c65413611ce" (UID: "d8680750-4437-46fe-bc00-4c65413611ce"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:52:11.660621 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.660597    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8680750-4437-46fe-bc00-4c65413611ce-kube-api-access-47qd5" (OuterVolumeSpecName: "kube-api-access-47qd5") pod "d8680750-4437-46fe-bc00-4c65413611ce" (UID: "d8680750-4437-46fe-bc00-4c65413611ce"). InnerVolumeSpecName "kube-api-access-47qd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 15:52:11.660701 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.660667    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d8680750-4437-46fe-bc00-4c65413611ce" (UID: "d8680750-4437-46fe-bc00-4c65413611ce"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:52:11.759505 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.759464    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:52:11.759505 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.759495    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47qd5\" (UniqueName: \"kubernetes.io/projected/d8680750-4437-46fe-bc00-4c65413611ce-kube-api-access-47qd5\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:52:11.759505 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.759510    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-oauth-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:52:11.759749 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.759522    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-trusted-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:52:11.759749 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.759534    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-console-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:52:11.759749 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.759545    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8680750-4437-46fe-bc00-4c65413611ce-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:52:11.759749 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:11.759556    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8680750-4437-46fe-bc00-4c65413611ce-console-oauth-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:52:12.335191 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:12.335158    2562 generic.go:356] "Generic (PLEG): container finished" podID="d8680750-4437-46fe-bc00-4c65413611ce" containerID="f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd" exitCode=0
Jul 23 15:52:12.335602 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:12.335202    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d788c457-gww6n"
Jul 23 15:52:12.335602 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:12.335256    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d788c457-gww6n" event={"ID":"d8680750-4437-46fe-bc00-4c65413611ce","Type":"ContainerDied","Data":"f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd"}
Jul 23 15:52:12.335602 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:12.335309    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d788c457-gww6n" event={"ID":"d8680750-4437-46fe-bc00-4c65413611ce","Type":"ContainerDied","Data":"2f61ecf561492bd6cd4331e7604674f0c5c413cb9bd410a56a96249d9c6c6ce5"}
Jul 23 15:52:12.335602 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:12.335331    2562 scope.go:117] "RemoveContainer" containerID="f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd"
Jul 23 15:52:12.343815 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:12.343784    2562 scope.go:117] "RemoveContainer" containerID="f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd"
Jul 23 15:52:12.344088 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:52:12.344070    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd\": container with ID starting with f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd not found: ID does not exist" containerID="f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd"
Jul 23 15:52:12.344143 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:12.344097    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd"} err="failed to get container status \"f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd\": rpc error: code = NotFound desc = could not find container \"f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd\": container with ID starting with f0ca41649d550442c133026f1cfd9fdb4b7d3483abb5578fbf8545c5ff4a55fd not found: ID does not exist"
Jul 23 15:52:12.356997 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:12.356971    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d788c457-gww6n"]
Jul 23 15:52:12.363514 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:12.363490    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d788c457-gww6n"]
Jul 23 15:52:13.252732 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:13.252700    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8680750-4437-46fe-bc00-4c65413611ce" path="/var/lib/kubelet/pods/d8680750-4437-46fe-bc00-4c65413611ce/volumes"
Jul 23 15:52:32.526132 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:32.526097    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:52:32.541885 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:32.541862    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:52:33.439377 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:52:33.439353    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 15:53:37.364765 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.364685    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77ddd8c9d8-c6lqr"]
Jul 23 15:53:37.365252 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.365085    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8680750-4437-46fe-bc00-4c65413611ce" containerName="console"
Jul 23 15:53:37.365252 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.365098    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8680750-4437-46fe-bc00-4c65413611ce" containerName="console"
Jul 23 15:53:37.365252 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.365159    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8680750-4437-46fe-bc00-4c65413611ce" containerName="console"
Jul 23 15:53:37.368212 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.368195    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.384025 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.383999    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77ddd8c9d8-c6lqr"]
Jul 23 15:53:37.501134 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.501096    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-service-ca\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.501134 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.501137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-oauth-config\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.501357 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.501199    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-console-config\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.501357 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.501221    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-trusted-ca-bundle\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.501357 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.501242    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-oauth-serving-cert\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.501357 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.501294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-serving-cert\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.501357 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.501338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp7c7\" (UniqueName: \"kubernetes.io/projected/ac19e41e-5d9b-4557-94c1-593730217095-kube-api-access-fp7c7\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.601952 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.601915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-trusted-ca-bundle\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.602112 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.601973    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-oauth-serving-cert\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.602173 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.602153    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-serving-cert\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.602224 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.602206    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp7c7\" (UniqueName: \"kubernetes.io/projected/ac19e41e-5d9b-4557-94c1-593730217095-kube-api-access-fp7c7\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.602267 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.602254    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-service-ca\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.602327 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.602295    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-oauth-config\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.602535 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.602493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-console-config\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.602797 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.602770    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-oauth-serving-cert\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.602925 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.602867    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-trusted-ca-bundle\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.603034 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.603010    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-service-ca\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.603153 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.603135    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-console-config\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.604774 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.604755    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-oauth-config\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.604868 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.604757    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-serving-cert\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.610163 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.610140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp7c7\" (UniqueName: \"kubernetes.io/projected/ac19e41e-5d9b-4557-94c1-593730217095-kube-api-access-fp7c7\") pod \"console-77ddd8c9d8-c6lqr\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") " pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.677274 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.677235    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:37.812661 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:37.812632    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77ddd8c9d8-c6lqr"]
Jul 23 15:53:37.814509 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:53:37.814475    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac19e41e_5d9b_4557_94c1_593730217095.slice/crio-a541afcd4eed067ed99ef3da7c9747589cd5b3f3f4612436c6a26c7005d9ce73 WatchSource:0}: Error finding container a541afcd4eed067ed99ef3da7c9747589cd5b3f3f4612436c6a26c7005d9ce73: Status 404 returned error can't find the container with id a541afcd4eed067ed99ef3da7c9747589cd5b3f3f4612436c6a26c7005d9ce73
Jul 23 15:53:38.711926 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:38.711887    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77ddd8c9d8-c6lqr" event={"ID":"ac19e41e-5d9b-4557-94c1-593730217095","Type":"ContainerStarted","Data":"d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821"}
Jul 23 15:53:38.712343 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:38.711970    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77ddd8c9d8-c6lqr" event={"ID":"ac19e41e-5d9b-4557-94c1-593730217095","Type":"ContainerStarted","Data":"a541afcd4eed067ed99ef3da7c9747589cd5b3f3f4612436c6a26c7005d9ce73"}
Jul 23 15:53:38.736674 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:38.736626    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77ddd8c9d8-c6lqr" podStartSLOduration=1.736611203 podStartE2EDuration="1.736611203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:53:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:53:38.735058574 +0000 UTC m=+281.939425969" watchObservedRunningTime="2026-07-23 15:53:38.736611203 +0000 UTC m=+281.940978595"
Jul 23 15:53:47.677815 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:47.677773    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:47.678257 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:47.677863    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:47.682701 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:47.682680    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:47.752741 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:47.752715    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 15:53:47.797345 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:47.797306    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-747488c4bb-wp7dl"]
Jul 23 15:53:57.100465 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:57.100432    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 15:53:57.103493 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:57.103469    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 15:53:57.107985 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:53:57.107965    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 15:54:12.818113 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:12.818070    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-747488c4bb-wp7dl" podUID="2d5c430c-3f14-4107-a861-78fd445bcb4f" containerName="console" containerID="cri-o://93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406" gracePeriod=15
Jul 23 15:54:13.065770 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.065743    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:54:13.120243 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120164    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-oauth-serving-cert\") pod \"2d5c430c-3f14-4107-a861-78fd445bcb4f\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") "
Jul 23 15:54:13.120243 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120216    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-oauth-config\") pod \"2d5c430c-3f14-4107-a861-78fd445bcb4f\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") "
Jul 23 15:54:13.120430 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120251    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-serving-cert\") pod \"2d5c430c-3f14-4107-a861-78fd445bcb4f\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") "
Jul 23 15:54:13.120430 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120354    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-config\") pod \"2d5c430c-3f14-4107-a861-78fd445bcb4f\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") "
Jul 23 15:54:13.120430 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120401    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-trusted-ca-bundle\") pod \"2d5c430c-3f14-4107-a861-78fd445bcb4f\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") "
Jul 23 15:54:13.120544 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120432    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-service-ca\") pod \"2d5c430c-3f14-4107-a861-78fd445bcb4f\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") "
Jul 23 15:54:13.120588 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120540    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzg6n\" (UniqueName: \"kubernetes.io/projected/2d5c430c-3f14-4107-a861-78fd445bcb4f-kube-api-access-wzg6n\") pod \"2d5c430c-3f14-4107-a861-78fd445bcb4f\" (UID: \"2d5c430c-3f14-4107-a861-78fd445bcb4f\") "
Jul 23 15:54:13.120710 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120677    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2d5c430c-3f14-4107-a861-78fd445bcb4f" (UID: "2d5c430c-3f14-4107-a861-78fd445bcb4f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:54:13.120818 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120743    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-config" (OuterVolumeSpecName: "console-config") pod "2d5c430c-3f14-4107-a861-78fd445bcb4f" (UID: "2d5c430c-3f14-4107-a861-78fd445bcb4f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:54:13.120892 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120866    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2d5c430c-3f14-4107-a861-78fd445bcb4f" (UID: "2d5c430c-3f14-4107-a861-78fd445bcb4f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:54:13.120951 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120938    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-oauth-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:54:13.121005 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120957    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:54:13.121005 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120971    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-trusted-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:54:13.121005 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.120969    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-service-ca" (OuterVolumeSpecName: "service-ca") pod "2d5c430c-3f14-4107-a861-78fd445bcb4f" (UID: "2d5c430c-3f14-4107-a861-78fd445bcb4f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 15:54:13.122447 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.122420    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2d5c430c-3f14-4107-a861-78fd445bcb4f" (UID: "2d5c430c-3f14-4107-a861-78fd445bcb4f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:54:13.122447 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.122439    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2d5c430c-3f14-4107-a861-78fd445bcb4f" (UID: "2d5c430c-3f14-4107-a861-78fd445bcb4f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 15:54:13.122581 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.122559    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d5c430c-3f14-4107-a861-78fd445bcb4f-kube-api-access-wzg6n" (OuterVolumeSpecName: "kube-api-access-wzg6n") pod "2d5c430c-3f14-4107-a861-78fd445bcb4f" (UID: "2d5c430c-3f14-4107-a861-78fd445bcb4f"). InnerVolumeSpecName "kube-api-access-wzg6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 15:54:13.222199 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.222160    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzg6n\" (UniqueName: \"kubernetes.io/projected/2d5c430c-3f14-4107-a861-78fd445bcb4f-kube-api-access-wzg6n\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:54:13.222199 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.222189    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-oauth-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:54:13.222199 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.222199    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d5c430c-3f14-4107-a861-78fd445bcb4f-console-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:54:13.222199 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.222208    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c430c-3f14-4107-a861-78fd445bcb4f-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 15:54:13.860237 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.860200    2562 generic.go:356] "Generic (PLEG): container finished" podID="2d5c430c-3f14-4107-a861-78fd445bcb4f" containerID="93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406" exitCode=0
Jul 23 15:54:13.860740 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.860247    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747488c4bb-wp7dl"
Jul 23 15:54:13.860740 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.860292    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747488c4bb-wp7dl" event={"ID":"2d5c430c-3f14-4107-a861-78fd445bcb4f","Type":"ContainerDied","Data":"93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406"}
Jul 23 15:54:13.860740 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.860350    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747488c4bb-wp7dl" event={"ID":"2d5c430c-3f14-4107-a861-78fd445bcb4f","Type":"ContainerDied","Data":"9af2e74234051701e8b072336e84d5f81afadcc891d57483a779bb7fa1d77df2"}
Jul 23 15:54:13.860740 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.860373    2562 scope.go:117] "RemoveContainer" containerID="93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406"
Jul 23 15:54:13.868763 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.868747    2562 scope.go:117] "RemoveContainer" containerID="93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406"
Jul 23 15:54:13.869087 ip-10-0-140-230 kubenswrapper[2562]: E0723 15:54:13.869062    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406\": container with ID starting with 93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406 not found: ID does not exist" containerID="93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406"
Jul 23 15:54:13.869194 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.869093    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406"} err="failed to get container status \"93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406\": rpc error: code = NotFound desc = could not find container \"93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406\": container with ID starting with 93d7e2d8723544f3b38394699a8e50e368e93ac0de0c7a11bd68d51423fd6406 not found: ID does not exist"
Jul 23 15:54:13.882840 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.882814    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-747488c4bb-wp7dl"]
Jul 23 15:54:13.888028 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:13.888002    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-747488c4bb-wp7dl"]
Jul 23 15:54:15.253583 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:15.253548    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d5c430c-3f14-4107-a861-78fd445bcb4f" path="/var/lib/kubelet/pods/2d5c430c-3f14-4107-a861-78fd445bcb4f/volumes"
Jul 23 15:54:21.774317 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:21.774287    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c7dmh"]
Jul 23 15:54:21.774705 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:21.774656    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d5c430c-3f14-4107-a861-78fd445bcb4f" containerName="console"
Jul 23 15:54:21.774705 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:21.774667    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d5c430c-3f14-4107-a861-78fd445bcb4f" containerName="console"
Jul 23 15:54:21.774779 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:21.774723    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d5c430c-3f14-4107-a861-78fd445bcb4f" containerName="console"
Jul 23 15:54:21.777629 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:21.777614    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c7dmh"
Jul 23 15:54:21.780432 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:21.780400    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-rld7c\""
Jul 23 15:54:21.780557 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:21.780439    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 23 15:54:21.780557 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:21.780463    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 23 15:54:21.789468 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:21.789447    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c7dmh"]
Jul 23 15:54:21.899054 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:21.899017    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84afd60d-6974-41a6-bd6f-08a067c86470-bound-sa-token\") pod \"cert-manager-55b4d954d5-c7dmh\" (UID: \"84afd60d-6974-41a6-bd6f-08a067c86470\") " pod="cert-manager/cert-manager-55b4d954d5-c7dmh"
Jul 23 15:54:21.899228 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:21.899106    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4pff\" (UniqueName: \"kubernetes.io/projected/84afd60d-6974-41a6-bd6f-08a067c86470-kube-api-access-k4pff\") pod \"cert-manager-55b4d954d5-c7dmh\" (UID: \"84afd60d-6974-41a6-bd6f-08a067c86470\") " pod="cert-manager/cert-manager-55b4d954d5-c7dmh"
Jul 23 15:54:22.000604 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:22.000570    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pff\" (UniqueName: \"kubernetes.io/projected/84afd60d-6974-41a6-bd6f-08a067c86470-kube-api-access-k4pff\") pod \"cert-manager-55b4d954d5-c7dmh\" (UID: \"84afd60d-6974-41a6-bd6f-08a067c86470\") " pod="cert-manager/cert-manager-55b4d954d5-c7dmh"
Jul 23 15:54:22.000773 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:22.000644    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84afd60d-6974-41a6-bd6f-08a067c86470-bound-sa-token\") pod \"cert-manager-55b4d954d5-c7dmh\" (UID: \"84afd60d-6974-41a6-bd6f-08a067c86470\") " pod="cert-manager/cert-manager-55b4d954d5-c7dmh"
Jul 23 15:54:22.010190 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:22.010156    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4pff\" (UniqueName: \"kubernetes.io/projected/84afd60d-6974-41a6-bd6f-08a067c86470-kube-api-access-k4pff\") pod \"cert-manager-55b4d954d5-c7dmh\" (UID: \"84afd60d-6974-41a6-bd6f-08a067c86470\") " pod="cert-manager/cert-manager-55b4d954d5-c7dmh"
Jul 23 15:54:22.010576 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:22.010557    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84afd60d-6974-41a6-bd6f-08a067c86470-bound-sa-token\") pod \"cert-manager-55b4d954d5-c7dmh\" (UID: \"84afd60d-6974-41a6-bd6f-08a067c86470\") " pod="cert-manager/cert-manager-55b4d954d5-c7dmh"
Jul 23 15:54:22.099306 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:22.099217    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c7dmh"
Jul 23 15:54:22.239317 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:22.239286    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c7dmh"]
Jul 23 15:54:22.241924 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:54:22.241898    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84afd60d_6974_41a6_bd6f_08a067c86470.slice/crio-1cc7fe021a3de7484a638838e023af502312d8e3d9a160c7084f1d1331b147f2 WatchSource:0}: Error finding container 1cc7fe021a3de7484a638838e023af502312d8e3d9a160c7084f1d1331b147f2: Status 404 returned error can't find the container with id 1cc7fe021a3de7484a638838e023af502312d8e3d9a160c7084f1d1331b147f2
Jul 23 15:54:22.243950 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:22.243932    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 15:54:22.900131 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:22.900095    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c7dmh" event={"ID":"84afd60d-6974-41a6-bd6f-08a067c86470","Type":"ContainerStarted","Data":"1cc7fe021a3de7484a638838e023af502312d8e3d9a160c7084f1d1331b147f2"}
Jul 23 15:54:24.997992 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:24.997961    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp"]
Jul 23 15:54:25.001762 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.001739    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp"
Jul 23 15:54:25.004740 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.004714    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 23 15:54:25.005971 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.005943    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-42cgg\""
Jul 23 15:54:25.006079 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.005941    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 23 15:54:25.012772 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.012753    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp"]
Jul 23 15:54:25.132400 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.132363    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1295e835-f6c9-4666-8598-c364388eb6b2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vpmmp\" (UID: \"1295e835-f6c9-4666-8598-c364388eb6b2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp"
Jul 23 15:54:25.132622 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.132478    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74fjx\" (UniqueName: \"kubernetes.io/projected/1295e835-f6c9-4666-8598-c364388eb6b2-kube-api-access-74fjx\") pod \"openshift-lws-operator-fd8ccff4c-vpmmp\" (UID: \"1295e835-f6c9-4666-8598-c364388eb6b2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp"
Jul 23 15:54:25.233732 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.233691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1295e835-f6c9-4666-8598-c364388eb6b2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vpmmp\" (UID: \"1295e835-f6c9-4666-8598-c364388eb6b2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp"
Jul 23 15:54:25.233919 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.233771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74fjx\" (UniqueName: \"kubernetes.io/projected/1295e835-f6c9-4666-8598-c364388eb6b2-kube-api-access-74fjx\") pod \"openshift-lws-operator-fd8ccff4c-vpmmp\" (UID: \"1295e835-f6c9-4666-8598-c364388eb6b2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp"
Jul 23 15:54:25.234212 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.234187    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1295e835-f6c9-4666-8598-c364388eb6b2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vpmmp\" (UID: \"1295e835-f6c9-4666-8598-c364388eb6b2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp"
Jul 23 15:54:25.243480 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.243449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74fjx\" (UniqueName: \"kubernetes.io/projected/1295e835-f6c9-4666-8598-c364388eb6b2-kube-api-access-74fjx\") pod \"openshift-lws-operator-fd8ccff4c-vpmmp\" (UID: \"1295e835-f6c9-4666-8598-c364388eb6b2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp"
Jul 23 15:54:25.314472 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:25.314388    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp"
Jul 23 15:54:26.021123 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:26.021099    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp"]
Jul 23 15:54:26.022962 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:54:26.022931    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1295e835_f6c9_4666_8598_c364388eb6b2.slice/crio-c622c6fd75505b46e340362c9daa5433d37dece366fb39065a9793d1261afd0f WatchSource:0}: Error finding container c622c6fd75505b46e340362c9daa5433d37dece366fb39065a9793d1261afd0f: Status 404 returned error can't find the container with id c622c6fd75505b46e340362c9daa5433d37dece366fb39065a9793d1261afd0f
Jul 23 15:54:26.921592 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:26.921549    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp" event={"ID":"1295e835-f6c9-4666-8598-c364388eb6b2","Type":"ContainerStarted","Data":"c622c6fd75505b46e340362c9daa5433d37dece366fb39065a9793d1261afd0f"}
Jul 23 15:54:26.923009 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:26.922988    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c7dmh" event={"ID":"84afd60d-6974-41a6-bd6f-08a067c86470","Type":"ContainerStarted","Data":"e3872c8a76213787495c33b189033de34519e58e5d76c9b71948b98759c2836a"}
Jul 23 15:54:26.940890 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:26.940841    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-c7dmh" podStartSLOduration=2.228003852 podStartE2EDuration="5.940824687s" totalImagesPullingTime="3.712820835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:54:26.940211716 +0000 UTC m=+330.144579112" watchObservedRunningTime="2026-07-23 15:54:26.940824687 +0000 UTC m=+330.145192077"
Jul 23 15:54:28.935311 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:28.935274    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp" event={"ID":"1295e835-f6c9-4666-8598-c364388eb6b2","Type":"ContainerStarted","Data":"c1b36f97a6f343cc7765006ecf56f7d3588e10580d6f49e4bc017464de6584f1"}
Jul 23 15:54:28.953656 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:28.953604    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vpmmp" podStartSLOduration=2.202594282 podStartE2EDuration="4.953586955s" totalImagesPullingTime="2.750992673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:54:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:54:28.952354329 +0000 UTC m=+332.156721745" watchObservedRunningTime="2026-07-23 15:54:28.953586955 +0000 UTC m=+332.157954347"
Jul 23 15:54:45.794739 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.794706    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"]
Jul 23 15:54:45.798325 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.798308    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.801286 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.801264    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 15:54:45.801574 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.801497    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9w9v5\""
Jul 23 15:54:45.801574 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.801518    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 15:54:45.801574 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.801534    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 15:54:45.801834 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.801581    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 15:54:45.831543 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.831511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d871218f-5819-44e3-9009-d2a196e35031-apiservice-cert\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.831706 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.831571    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc5kb\" (UniqueName: \"kubernetes.io/projected/d871218f-5819-44e3-9009-d2a196e35031-kube-api-access-zc5kb\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.831706 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.831628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d871218f-5819-44e3-9009-d2a196e35031-manifests\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.831828 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.831711    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d871218f-5819-44e3-9009-d2a196e35031-webhook-cert\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.831828 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.831743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d871218f-5819-44e3-9009-d2a196e35031-tmp\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.908937 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.908908    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"]
Jul 23 15:54:45.932195 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.932164    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d871218f-5819-44e3-9009-d2a196e35031-manifests\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.932381 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.932209    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d871218f-5819-44e3-9009-d2a196e35031-webhook-cert\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.932381 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.932228    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d871218f-5819-44e3-9009-d2a196e35031-tmp\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.932381 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.932275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d871218f-5819-44e3-9009-d2a196e35031-apiservice-cert\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.932381 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.932316    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc5kb\" (UniqueName: \"kubernetes.io/projected/d871218f-5819-44e3-9009-d2a196e35031-kube-api-access-zc5kb\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.932690 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.932661    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d871218f-5819-44e3-9009-d2a196e35031-manifests\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.934565 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.934542    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d871218f-5819-44e3-9009-d2a196e35031-tmp\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.934760 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.934741    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d871218f-5819-44e3-9009-d2a196e35031-apiservice-cert\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.934820 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.934741    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d871218f-5819-44e3-9009-d2a196e35031-webhook-cert\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:45.946721 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:45.946695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc5kb\" (UniqueName: \"kubernetes.io/projected/d871218f-5819-44e3-9009-d2a196e35031-kube-api-access-zc5kb\") pod \"opendatahub-operator-controller-manager-646b5d5c79-tdxr9\" (UID: \"d871218f-5819-44e3-9009-d2a196e35031\") " pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:46.109118 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:46.109031    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:46.259674 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:46.259580    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"]
Jul 23 15:54:46.265639 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:54:46.265607    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd871218f_5819_44e3_9009_d2a196e35031.slice/crio-97e262134adac89adaad00134d1e68b9590311213b7ebbf003b8891b6fdcb951 WatchSource:0}: Error finding container 97e262134adac89adaad00134d1e68b9590311213b7ebbf003b8891b6fdcb951: Status 404 returned error can't find the container with id 97e262134adac89adaad00134d1e68b9590311213b7ebbf003b8891b6fdcb951
Jul 23 15:54:47.020846 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:47.020789    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9" event={"ID":"d871218f-5819-44e3-9009-d2a196e35031","Type":"ContainerStarted","Data":"97e262134adac89adaad00134d1e68b9590311213b7ebbf003b8891b6fdcb951"}
Jul 23 15:54:50.038347 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:50.038307    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9" event={"ID":"d871218f-5819-44e3-9009-d2a196e35031","Type":"ContainerStarted","Data":"946b8fe1e5e42d2e5610dbe1b0c0df3bad25065f37d812d2d58b3ec0d6f748df"}
Jul 23 15:54:51.044892 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:51.044859    2562 generic.go:356] "Generic (PLEG): container finished" podID="d871218f-5819-44e3-9009-d2a196e35031" containerID="946b8fe1e5e42d2e5610dbe1b0c0df3bad25065f37d812d2d58b3ec0d6f748df" exitCode=0
Jul 23 15:54:51.045274 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:51.044922    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9" event={"ID":"d871218f-5819-44e3-9009-d2a196e35031","Type":"ContainerDied","Data":"946b8fe1e5e42d2e5610dbe1b0c0df3bad25065f37d812d2d58b3ec0d6f748df"}
Jul 23 15:54:52.053162 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:52.053057    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9" event={"ID":"d871218f-5819-44e3-9009-d2a196e35031","Type":"ContainerStarted","Data":"93280f0efb59e0fb92bddf4e5cc5324eef7c1e39319d1f2af12eae83be9053f5"}
Jul 23 15:54:52.053606 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:52.053242    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:54:52.095950 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:54:52.095890    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9" podStartSLOduration=2.1464393 podStartE2EDuration="7.095868425s" totalImagesPullingTime="3.873324187s" totalInitContainerRuntime="1.076104938s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:54:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:54:52.092901866 +0000 UTC m=+355.297269261" watchObservedRunningTime="2026-07-23 15:54:52.095868425 +0000 UTC m=+355.300235819"
Jul 23 15:55:03.060463 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:03.060426    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-646b5d5c79-tdxr9"
Jul 23 15:55:05.391932 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.391900    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"]
Jul 23 15:55:05.405532 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.405499    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.416724 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.416672    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zlv99\""
Jul 23 15:55:05.417146 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.417131    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 23 15:55:05.420252 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.420225    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"]
Jul 23 15:55:05.422389 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.422368    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 23 15:55:05.422525 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.422437    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 23 15:55:05.510591 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.510557    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-cert\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.510775 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.510614    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-manager-config\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.510775 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.510678    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-metrics-cert\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.510775 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.510759    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcdgt\" (UniqueName: \"kubernetes.io/projected/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-kube-api-access-dcdgt\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.611410 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.611367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-manager-config\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.611627 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.611423    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-metrics-cert\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.611627 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.611476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcdgt\" (UniqueName: \"kubernetes.io/projected/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-kube-api-access-dcdgt\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.611627 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.611562    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-cert\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.612228 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.612198    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-manager-config\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.614206 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.614178    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-metrics-cert\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.614333 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.614189    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-cert\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.667786 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.667695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcdgt\" (UniqueName: \"kubernetes.io/projected/6c8d0ae6-dd1b-4844-9f13-8dcd175d7373-kube-api-access-dcdgt\") pod \"lws-controller-manager-5c5865ccbd-gph44\" (UID: \"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373\") " pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.715446 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.715413    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:05.908819 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:55:05.908771    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c8d0ae6_dd1b_4844_9f13_8dcd175d7373.slice/crio-14c187db2c376e7f03664f2cc95a38129b5cead6ff78f9c4160abb9d5ebc2b5f WatchSource:0}: Error finding container 14c187db2c376e7f03664f2cc95a38129b5cead6ff78f9c4160abb9d5ebc2b5f: Status 404 returned error can't find the container with id 14c187db2c376e7f03664f2cc95a38129b5cead6ff78f9c4160abb9d5ebc2b5f
Jul 23 15:55:05.931630 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:05.931606    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"]
Jul 23 15:55:06.124907 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.124869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44" event={"ID":"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373","Type":"ContainerStarted","Data":"14c187db2c376e7f03664f2cc95a38129b5cead6ff78f9c4160abb9d5ebc2b5f"}
Jul 23 15:55:06.233138 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.233051    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"]
Jul 23 15:55:06.238492 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.238465    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"
Jul 23 15:55:06.242813 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.242773    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 15:55:06.242975 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.242831    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 15:55:06.244033 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.244012    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-k8sm9\""
Jul 23 15:55:06.244152 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.244012    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 15:55:06.244152 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.244054    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 15:55:06.269563 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.269517    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"]
Jul 23 15:55:06.317977 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.317944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7158c456-141f-4e29-8ce3-711b500e2b1a-tls-certs\") pod \"kube-auth-proxy-5fb5dcd56-rwpjm\" (UID: \"7158c456-141f-4e29-8ce3-711b500e2b1a\") " pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"
Jul 23 15:55:06.317977 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.317979    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdtr6\" (UniqueName: \"kubernetes.io/projected/7158c456-141f-4e29-8ce3-711b500e2b1a-kube-api-access-xdtr6\") pod \"kube-auth-proxy-5fb5dcd56-rwpjm\" (UID: \"7158c456-141f-4e29-8ce3-711b500e2b1a\") " pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"
Jul 23 15:55:06.318190 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.318067    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7158c456-141f-4e29-8ce3-711b500e2b1a-tmp\") pod \"kube-auth-proxy-5fb5dcd56-rwpjm\" (UID: \"7158c456-141f-4e29-8ce3-711b500e2b1a\") " pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"
Jul 23 15:55:06.419242 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.419207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7158c456-141f-4e29-8ce3-711b500e2b1a-tls-certs\") pod \"kube-auth-proxy-5fb5dcd56-rwpjm\" (UID: \"7158c456-141f-4e29-8ce3-711b500e2b1a\") " pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"
Jul 23 15:55:06.419242 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.419246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdtr6\" (UniqueName: \"kubernetes.io/projected/7158c456-141f-4e29-8ce3-711b500e2b1a-kube-api-access-xdtr6\") pod \"kube-auth-proxy-5fb5dcd56-rwpjm\" (UID: \"7158c456-141f-4e29-8ce3-711b500e2b1a\") " pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"
Jul 23 15:55:06.419650 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.419297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7158c456-141f-4e29-8ce3-711b500e2b1a-tmp\") pod \"kube-auth-proxy-5fb5dcd56-rwpjm\" (UID: \"7158c456-141f-4e29-8ce3-711b500e2b1a\") " pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"
Jul 23 15:55:06.421635 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.421611    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7158c456-141f-4e29-8ce3-711b500e2b1a-tmp\") pod \"kube-auth-proxy-5fb5dcd56-rwpjm\" (UID: \"7158c456-141f-4e29-8ce3-711b500e2b1a\") " pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"
Jul 23 15:55:06.421791 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.421776    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7158c456-141f-4e29-8ce3-711b500e2b1a-tls-certs\") pod \"kube-auth-proxy-5fb5dcd56-rwpjm\" (UID: \"7158c456-141f-4e29-8ce3-711b500e2b1a\") " pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"
Jul 23 15:55:06.441407 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.441377    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdtr6\" (UniqueName: \"kubernetes.io/projected/7158c456-141f-4e29-8ce3-711b500e2b1a-kube-api-access-xdtr6\") pod \"kube-auth-proxy-5fb5dcd56-rwpjm\" (UID: \"7158c456-141f-4e29-8ce3-711b500e2b1a\") " pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"
Jul 23 15:55:06.549094 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.549011    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"
Jul 23 15:55:06.750248 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:06.750211    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm"]
Jul 23 15:55:06.755494 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:55:06.755466    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7158c456_141f_4e29_8ce3_711b500e2b1a.slice/crio-536b356657340ef1eeebf2b4a331ff6e32483bb370091ff8fda927652b222ac7 WatchSource:0}: Error finding container 536b356657340ef1eeebf2b4a331ff6e32483bb370091ff8fda927652b222ac7: Status 404 returned error can't find the container with id 536b356657340ef1eeebf2b4a331ff6e32483bb370091ff8fda927652b222ac7
Jul 23 15:55:07.133578 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:07.133521    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm" event={"ID":"7158c456-141f-4e29-8ce3-711b500e2b1a","Type":"ContainerStarted","Data":"536b356657340ef1eeebf2b4a331ff6e32483bb370091ff8fda927652b222ac7"}
Jul 23 15:55:08.142453 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:08.142417    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44" event={"ID":"6c8d0ae6-dd1b-4844-9f13-8dcd175d7373","Type":"ContainerStarted","Data":"26fcc1e0dd6d8dd3c4613a7c5ef0655af277b73e07f2aef7a820bc275fb39245"}
Jul 23 15:55:08.142933 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:08.142608    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:55:08.215187 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:08.215135    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44" podStartSLOduration=1.511726596 podStartE2EDuration="3.215117226s" totalImagesPullingTime="1.70339063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:55:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:55:08.214531138 +0000 UTC m=+371.418898530" watchObservedRunningTime="2026-07-23 15:55:08.215117226 +0000 UTC m=+371.419484692"
Jul 23 15:55:10.156004 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:10.155908    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm" event={"ID":"7158c456-141f-4e29-8ce3-711b500e2b1a","Type":"ContainerStarted","Data":"e2be1cbda8898cf964d74645711e6c9089e8f5ff9082f640f1bd44ac24dead1d"}
Jul 23 15:55:10.174618 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:10.174573    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5fb5dcd56-rwpjm" podStartSLOduration=1.064260023 podStartE2EDuration="4.174556866s" totalImagesPullingTime="3.110296843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:55:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:55:10.173039814 +0000 UTC m=+373.377407211" watchObservedRunningTime="2026-07-23 15:55:10.174556866 +0000 UTC m=+373.378924260"
Jul 23 15:55:19.150990 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:55:19.150953    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c5865ccbd-gph44"
Jul 23 15:56:52.075856 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.075816    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p"]
Jul 23 15:56:52.079927 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.079895    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p"
Jul 23 15:56:52.089611 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.089590    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 23 15:56:52.089611 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.089598    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 15:56:52.090187 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.090167    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 15:56:52.090187 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.090178    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dsvmf\""
Jul 23 15:56:52.103747 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.103712    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck64n\" (UniqueName: \"kubernetes.io/projected/f4617220-9992-4496-98e1-b0c489035f1b-kube-api-access-ck64n\") pod \"dns-operator-controller-manager-864dd78c4c-r8k4p\" (UID: \"f4617220-9992-4496-98e1-b0c489035f1b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p"
Jul 23 15:56:52.108259 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.108235    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p"]
Jul 23 15:56:52.211362 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.206036    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck64n\" (UniqueName: \"kubernetes.io/projected/f4617220-9992-4496-98e1-b0c489035f1b-kube-api-access-ck64n\") pod \"dns-operator-controller-manager-864dd78c4c-r8k4p\" (UID: \"f4617220-9992-4496-98e1-b0c489035f1b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p"
Jul 23 15:56:52.237213 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.237185    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck64n\" (UniqueName: \"kubernetes.io/projected/f4617220-9992-4496-98e1-b0c489035f1b-kube-api-access-ck64n\") pod \"dns-operator-controller-manager-864dd78c4c-r8k4p\" (UID: \"f4617220-9992-4496-98e1-b0c489035f1b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p"
Jul 23 15:56:52.391350 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.391255    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p"
Jul 23 15:56:52.580013 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:56:52.579988    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4617220_9992_4496_98e1_b0c489035f1b.slice/crio-be786e90d53ba4350f3440483a65bf036eb3d962a40a99c1e6e610d9b18b4ecc WatchSource:0}: Error finding container be786e90d53ba4350f3440483a65bf036eb3d962a40a99c1e6e610d9b18b4ecc: Status 404 returned error can't find the container with id be786e90d53ba4350f3440483a65bf036eb3d962a40a99c1e6e610d9b18b4ecc
Jul 23 15:56:52.580422 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.580398    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p"]
Jul 23 15:56:52.645685 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:52.645601    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p" event={"ID":"f4617220-9992-4496-98e1-b0c489035f1b","Type":"ContainerStarted","Data":"be786e90d53ba4350f3440483a65bf036eb3d962a40a99c1e6e610d9b18b4ecc"}
Jul 23 15:56:53.994475 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:53.994439    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"]
Jul 23 15:56:53.997730 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:53.997711    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"
Jul 23 15:56:54.001231 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.001204    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 23 15:56:54.001510 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.001386    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8257l\""
Jul 23 15:56:54.001510 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.001410    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 23 15:56:54.024516 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.024438    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6ztr\" (UniqueName: \"kubernetes.io/projected/ec736c46-6305-4a0d-ae9a-5b83ac485177-kube-api-access-t6ztr\") pod \"kuadrant-console-plugin-66b5774985-mbsjf\" (UID: \"ec736c46-6305-4a0d-ae9a-5b83ac485177\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"
Jul 23 15:56:54.024783 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.024595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec736c46-6305-4a0d-ae9a-5b83ac485177-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mbsjf\" (UID: \"ec736c46-6305-4a0d-ae9a-5b83ac485177\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"
Jul 23 15:56:54.024783 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.024734    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ec736c46-6305-4a0d-ae9a-5b83ac485177-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mbsjf\" (UID: \"ec736c46-6305-4a0d-ae9a-5b83ac485177\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"
Jul 23 15:56:54.027597 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.027571    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"]
Jul 23 15:56:54.126134 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.126091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ec736c46-6305-4a0d-ae9a-5b83ac485177-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mbsjf\" (UID: \"ec736c46-6305-4a0d-ae9a-5b83ac485177\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"
Jul 23 15:56:54.126134 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.126141    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6ztr\" (UniqueName: \"kubernetes.io/projected/ec736c46-6305-4a0d-ae9a-5b83ac485177-kube-api-access-t6ztr\") pod \"kuadrant-console-plugin-66b5774985-mbsjf\" (UID: \"ec736c46-6305-4a0d-ae9a-5b83ac485177\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"
Jul 23 15:56:54.126474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.126168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec736c46-6305-4a0d-ae9a-5b83ac485177-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mbsjf\" (UID: \"ec736c46-6305-4a0d-ae9a-5b83ac485177\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"
Jul 23 15:56:54.126750 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.126727    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ec736c46-6305-4a0d-ae9a-5b83ac485177-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mbsjf\" (UID: \"ec736c46-6305-4a0d-ae9a-5b83ac485177\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"
Jul 23 15:56:54.128344 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.128321    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec736c46-6305-4a0d-ae9a-5b83ac485177-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mbsjf\" (UID: \"ec736c46-6305-4a0d-ae9a-5b83ac485177\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"
Jul 23 15:56:54.145437 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.145403    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6ztr\" (UniqueName: \"kubernetes.io/projected/ec736c46-6305-4a0d-ae9a-5b83ac485177-kube-api-access-t6ztr\") pod \"kuadrant-console-plugin-66b5774985-mbsjf\" (UID: \"ec736c46-6305-4a0d-ae9a-5b83ac485177\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"
Jul 23 15:56:54.307888 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.307814    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"
Jul 23 15:56:54.801585 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:54.801530    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf"]
Jul 23 15:56:54.804406 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:56:54.804382    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec736c46_6305_4a0d_ae9a_5b83ac485177.slice/crio-9797154a6884db40848173400be88a2a683d43501b77f61cc2c7fe3bdee4e24a WatchSource:0}: Error finding container 9797154a6884db40848173400be88a2a683d43501b77f61cc2c7fe3bdee4e24a: Status 404 returned error can't find the container with id 9797154a6884db40848173400be88a2a683d43501b77f61cc2c7fe3bdee4e24a
Jul 23 15:56:55.663909 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:55.663839    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf" event={"ID":"ec736c46-6305-4a0d-ae9a-5b83ac485177","Type":"ContainerStarted","Data":"9797154a6884db40848173400be88a2a683d43501b77f61cc2c7fe3bdee4e24a"}
Jul 23 15:56:55.666293 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:55.666239    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p" event={"ID":"f4617220-9992-4496-98e1-b0c489035f1b","Type":"ContainerStarted","Data":"364963257c9beca282175e5f01c766e1a80aecaf4b24c1423b015b59e7ddd42b"}
Jul 23 15:56:55.666445 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:55.666415    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p"
Jul 23 15:56:55.710771 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:56:55.710708    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p" podStartSLOduration=1.611526344 podStartE2EDuration="3.710684196s" totalImagesPullingTime="2.099157852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:56:55.708495311 +0000 UTC m=+478.912862705" watchObservedRunningTime="2026-07-23 15:56:55.710684196 +0000 UTC m=+478.915051592"
Jul 23 15:57:06.676061 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:06.676029    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8k4p"
Jul 23 15:57:10.738002 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:10.737966    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"]
Jul 23 15:57:10.800718 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:10.800687    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"]
Jul 23 15:57:10.800923 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:10.800816    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 15:57:10.804474 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:10.804441    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-m2dnw\""
Jul 23 15:57:10.891743 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:10.891709    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hmrpr\" (UID: \"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 15:57:10.891743 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:10.891755    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5tnw\" (UniqueName: \"kubernetes.io/projected/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-kube-api-access-h5tnw\") pod \"kuadrant-operator-controller-manager-86878c769f-hmrpr\" (UID: \"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 15:57:10.993035 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:10.992933    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hmrpr\" (UID: \"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 15:57:10.993035 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:10.992990    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5tnw\" (UniqueName: \"kubernetes.io/projected/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-kube-api-access-h5tnw\") pod \"kuadrant-operator-controller-manager-86878c769f-hmrpr\" (UID: \"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 15:57:10.993665 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:10.993639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hmrpr\" (UID: \"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 15:57:11.006847 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:11.006793    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5tnw\" (UniqueName: \"kubernetes.io/projected/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-kube-api-access-h5tnw\") pod \"kuadrant-operator-controller-manager-86878c769f-hmrpr\" (UID: \"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 15:57:11.112609 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:11.112562    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 15:57:16.977957 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:16.977934    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"]
Jul 23 15:57:16.980461 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:57:16.980436    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad371a3c_60fa_4aa0_87cc_dea03c1b86f6.slice/crio-6ad09115e77f7948de7aae79ae1e4bba26a04f2781509136ad65a49b61f366f1 WatchSource:0}: Error finding container 6ad09115e77f7948de7aae79ae1e4bba26a04f2781509136ad65a49b61f366f1: Status 404 returned error can't find the container with id 6ad09115e77f7948de7aae79ae1e4bba26a04f2781509136ad65a49b61f366f1
Jul 23 15:57:17.801895 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:17.801857    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf" event={"ID":"ec736c46-6305-4a0d-ae9a-5b83ac485177","Type":"ContainerStarted","Data":"6ce11a7e28074d5478d8b17faccd4cd109292e9cb5c0c89b6dbddb5c8159647d"}
Jul 23 15:57:17.803382 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:17.803357    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr" event={"ID":"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6","Type":"ContainerStarted","Data":"6ad09115e77f7948de7aae79ae1e4bba26a04f2781509136ad65a49b61f366f1"}
Jul 23 15:57:17.870459 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:17.870335    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mbsjf" podStartSLOduration=2.772218726 podStartE2EDuration="24.870309436s" totalImagesPullingTime="22.09809071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:57:17.868933182 +0000 UTC m=+501.073300574" watchObservedRunningTime="2026-07-23 15:57:17.870309436 +0000 UTC m=+501.074676830"
Jul 23 15:57:21.825208 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:21.825172    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr" event={"ID":"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6","Type":"ContainerStarted","Data":"591ef56cfd91d8c42b873236e143e9fd5eb13087212d3490934fd791f11e589e"}
Jul 23 15:57:21.825600 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:21.825290    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 15:57:32.834752 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:32.834718    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 15:57:32.859935 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:57:32.859885    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr" podStartSLOduration=18.450714137 podStartE2EDuration="22.859867433s" totalImagesPullingTime="4.409153296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:57:21.86820546 +0000 UTC m=+505.072572865" watchObservedRunningTime="2026-07-23 15:57:32.859867433 +0000 UTC m=+516.064234825"
Jul 23 15:58:57.134213 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:58:57.134182    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 15:58:57.135115 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:58:57.135082    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 15:59:36.044533 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.044443    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"]
Jul 23 15:59:36.047974 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.047958    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.051101 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.051078    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 15:59:36.051246 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.051161    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vlp5j\""
Jul 23 15:59:36.052403 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.052384    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 15:59:36.052512 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.052384    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 23 15:59:36.061076 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.061051    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"]
Jul 23 15:59:36.125290 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.125256    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.125290 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.125291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.125506 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.125311    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.125506 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.125327    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfrxv\" (UniqueName: \"kubernetes.io/projected/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-kube-api-access-sfrxv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.125506 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.125422    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.125506 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.125456    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.226616 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.226583    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.226793 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.226625    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.226793 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.226681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.227006 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.226824    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.227006 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.226861    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.227006 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.226885    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfrxv\" (UniqueName: \"kubernetes.io/projected/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-kube-api-access-sfrxv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.227216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.227098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.227216 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.227160    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.227341 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.227227    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.229074 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.229047    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.229409 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.229390    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.238772 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.238736    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfrxv\" (UniqueName: \"kubernetes.io/projected/4e416ac9-c394-4517-b5b9-359ef1f4e8cf-kube-api-access-sfrxv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc\" (UID: \"4e416ac9-c394-4517-b5b9-359ef1f4e8cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.358744 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.358660    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:36.505309 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.505275    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"]
Jul 23 15:59:36.508433 ip-10-0-140-230 kubenswrapper[2562]: W0723 15:59:36.508404    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e416ac9_c394_4517_b5b9_359ef1f4e8cf.slice/crio-d086b7c0869638e6a2b3d1a5cec771a4b16adbe220631e0a6518c87375838459 WatchSource:0}: Error finding container d086b7c0869638e6a2b3d1a5cec771a4b16adbe220631e0a6518c87375838459: Status 404 returned error can't find the container with id d086b7c0869638e6a2b3d1a5cec771a4b16adbe220631e0a6518c87375838459
Jul 23 15:59:36.509748 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:36.509732    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 15:59:37.500259 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:37.500218    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc" event={"ID":"4e416ac9-c394-4517-b5b9-359ef1f4e8cf","Type":"ContainerStarted","Data":"d086b7c0869638e6a2b3d1a5cec771a4b16adbe220631e0a6518c87375838459"}
Jul 23 15:59:38.507782 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:38.507745    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc" event={"ID":"4e416ac9-c394-4517-b5b9-359ef1f4e8cf","Type":"ContainerStarted","Data":"83f1e619e3a19b6286c2e59c9f74db28968b1f62b1a1ffef052d54e2fea75f99"}
Jul 23 15:59:38.535505 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:38.535452    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc" podStartSLOduration=1.380421932 podStartE2EDuration="2.535434754s" totalImagesPullingTime="1.155012822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:59:38.534135592 +0000 UTC m=+641.738502984" watchObservedRunningTime="2026-07-23 15:59:38.535434754 +0000 UTC m=+641.739802146"
Jul 23 15:59:46.359607 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:46.359566    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:46.360031 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:46.359644    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:46.372491 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:46.372464    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 15:59:46.557901 ip-10-0-140-230 kubenswrapper[2562]: I0723 15:59:46.557875    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc"
Jul 23 16:00:00.151738 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:00.151706    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747040-mt6w6"]
Jul 23 16:00:00.155180 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:00.155163    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747040-mt6w6"
Jul 23 16:00:00.158077 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:00.158055    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w6jwk\""
Jul 23 16:00:00.163784 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:00.163761    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747040-mt6w6"]
Jul 23 16:00:00.167370 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:00.167348    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747040-mt6w6"
Jul 23 16:00:00.311625 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:00.311594    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747040-mt6w6"]
Jul 23 16:00:00.314386 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:00:00.314354    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d416850_9072_4567_94ea_a795495a810b.slice/crio-dda66d5dfae2d82a5adf0e823320da147405be23b37d29a0ace539dc3b3123ab WatchSource:0}: Error finding container dda66d5dfae2d82a5adf0e823320da147405be23b37d29a0ace539dc3b3123ab: Status 404 returned error can't find the container with id dda66d5dfae2d82a5adf0e823320da147405be23b37d29a0ace539dc3b3123ab
Jul 23 16:00:00.616279 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:00.616243    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747040-mt6w6" event={"ID":"7d416850-9072-4567-94ea-a795495a810b","Type":"ContainerStarted","Data":"dda66d5dfae2d82a5adf0e823320da147405be23b37d29a0ace539dc3b3123ab"}
Jul 23 16:00:03.633259 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:03.633223    2562 generic.go:356] "Generic (PLEG): container finished" podID="7d416850-9072-4567-94ea-a795495a810b" containerID="e45867a77ec28408316923f4f3f40e495799031bbaf36217ce5cfa0d293c9701" exitCode=0
Jul 23 16:00:03.633630 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:03.633266    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747040-mt6w6" event={"ID":"7d416850-9072-4567-94ea-a795495a810b","Type":"ContainerDied","Data":"e45867a77ec28408316923f4f3f40e495799031bbaf36217ce5cfa0d293c9701"}
Jul 23 16:00:04.777763 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:04.777739    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747040-mt6w6"
Jul 23 16:00:05.647942 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:05.647859    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747040-mt6w6" event={"ID":"7d416850-9072-4567-94ea-a795495a810b","Type":"ContainerDied","Data":"dda66d5dfae2d82a5adf0e823320da147405be23b37d29a0ace539dc3b3123ab"}
Jul 23 16:00:05.647942 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:05.647885    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747040-mt6w6"
Jul 23 16:00:05.647942 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:00:05.647906    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dda66d5dfae2d82a5adf0e823320da147405be23b37d29a0ace539dc3b3123ab"
Jul 23 16:01:38.233934 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.233898    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77dfddf987-rqlwq"]
Jul 23 16:01:38.234420 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.234288    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d416850-9072-4567-94ea-a795495a810b" containerName="cleanup"
Jul 23 16:01:38.234420 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.234300    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d416850-9072-4567-94ea-a795495a810b" containerName="cleanup"
Jul 23 16:01:38.234420 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.234372    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d416850-9072-4567-94ea-a795495a810b" containerName="cleanup"
Jul 23 16:01:38.237018 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.236998    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.254944 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.254916    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77dfddf987-rqlwq"]
Jul 23 16:01:38.351543 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.351505    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-oauth-serving-cert\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.351742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.351569    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e38d3077-86da-4f74-852b-4f84fe4c8a74-console-serving-cert\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.351742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.351616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-console-config\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.351742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.351637    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-trusted-ca-bundle\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.351742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.351660    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf548\" (UniqueName: \"kubernetes.io/projected/e38d3077-86da-4f74-852b-4f84fe4c8a74-kube-api-access-tf548\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.351742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.351676    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-service-ca\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.351742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.351706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e38d3077-86da-4f74-852b-4f84fe4c8a74-console-oauth-config\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.452420 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.452388    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-oauth-serving-cert\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.452598 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.452445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e38d3077-86da-4f74-852b-4f84fe4c8a74-console-serving-cert\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.452598 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.452474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-console-config\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.452598 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.452496    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-trusted-ca-bundle\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.452598 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.452512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf548\" (UniqueName: \"kubernetes.io/projected/e38d3077-86da-4f74-852b-4f84fe4c8a74-kube-api-access-tf548\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.452598 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.452533    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-service-ca\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.452890 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.452758    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e38d3077-86da-4f74-852b-4f84fe4c8a74-console-oauth-config\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.453323 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.453295    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-console-config\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.453323 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.453313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-service-ca\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.453536 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.453295    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-oauth-serving-cert\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.453536 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.453452    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38d3077-86da-4f74-852b-4f84fe4c8a74-trusted-ca-bundle\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.455085 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.455061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e38d3077-86da-4f74-852b-4f84fe4c8a74-console-serving-cert\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.455085 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.455081    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e38d3077-86da-4f74-852b-4f84fe4c8a74-console-oauth-config\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.462115 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.462089    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf548\" (UniqueName: \"kubernetes.io/projected/e38d3077-86da-4f74-852b-4f84fe4c8a74-kube-api-access-tf548\") pod \"console-77dfddf987-rqlwq\" (UID: \"e38d3077-86da-4f74-852b-4f84fe4c8a74\") " pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.547138 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.547044    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:38.691057 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:38.691031    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77dfddf987-rqlwq"]
Jul 23 16:01:38.692973 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:01:38.692945    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode38d3077_86da_4f74_852b_4f84fe4c8a74.slice/crio-b5f683d800e2e62245e03104abc002bb8575895b5ed99c191af74491921ff730 WatchSource:0}: Error finding container b5f683d800e2e62245e03104abc002bb8575895b5ed99c191af74491921ff730: Status 404 returned error can't find the container with id b5f683d800e2e62245e03104abc002bb8575895b5ed99c191af74491921ff730
Jul 23 16:01:39.122585 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:39.122549    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dfddf987-rqlwq" event={"ID":"e38d3077-86da-4f74-852b-4f84fe4c8a74","Type":"ContainerStarted","Data":"93b3c7bc86aad2d9eaca0fc0ad868928ddee1d8a03152b1898b47db89abcc0a7"}
Jul 23 16:01:39.122758 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:39.122600    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dfddf987-rqlwq" event={"ID":"e38d3077-86da-4f74-852b-4f84fe4c8a74","Type":"ContainerStarted","Data":"b5f683d800e2e62245e03104abc002bb8575895b5ed99c191af74491921ff730"}
Jul 23 16:01:39.144570 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:39.144511    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77dfddf987-rqlwq" podStartSLOduration=1.144489688 podStartE2EDuration="1.144489688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:01:39.142735434 +0000 UTC m=+762.347102827" watchObservedRunningTime="2026-07-23 16:01:39.144489688 +0000 UTC m=+762.348857182"
Jul 23 16:01:48.547823 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:48.547761    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:48.548331 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:48.547869    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:48.552610 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:48.552586    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:49.175692 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:49.175666    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77dfddf987-rqlwq"
Jul 23 16:01:49.262495 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:01:49.262466    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77ddd8c9d8-c6lqr"]
Jul 23 16:02:14.283782 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.283739    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77ddd8c9d8-c6lqr" podUID="ac19e41e-5d9b-4557-94c1-593730217095" containerName="console" containerID="cri-o://d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821" gracePeriod=15
Jul 23 16:02:14.537838 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.537741    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 16:02:14.617553 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.617519    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-console-config\") pod \"ac19e41e-5d9b-4557-94c1-593730217095\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") "
Jul 23 16:02:14.617768 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.617572    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp7c7\" (UniqueName: \"kubernetes.io/projected/ac19e41e-5d9b-4557-94c1-593730217095-kube-api-access-fp7c7\") pod \"ac19e41e-5d9b-4557-94c1-593730217095\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") "
Jul 23 16:02:14.617768 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.617605    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-service-ca\") pod \"ac19e41e-5d9b-4557-94c1-593730217095\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") "
Jul 23 16:02:14.617768 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.617638    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-oauth-serving-cert\") pod \"ac19e41e-5d9b-4557-94c1-593730217095\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") "
Jul 23 16:02:14.617768 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.617738    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-serving-cert\") pod \"ac19e41e-5d9b-4557-94c1-593730217095\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") "
Jul 23 16:02:14.618049 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.617862    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-oauth-config\") pod \"ac19e41e-5d9b-4557-94c1-593730217095\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") "
Jul 23 16:02:14.618049 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.617898    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-trusted-ca-bundle\") pod \"ac19e41e-5d9b-4557-94c1-593730217095\" (UID: \"ac19e41e-5d9b-4557-94c1-593730217095\") "
Jul 23 16:02:14.618153 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.618039    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-console-config" (OuterVolumeSpecName: "console-config") pod "ac19e41e-5d9b-4557-94c1-593730217095" (UID: "ac19e41e-5d9b-4557-94c1-593730217095"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:02:14.618153 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.618049    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ac19e41e-5d9b-4557-94c1-593730217095" (UID: "ac19e41e-5d9b-4557-94c1-593730217095"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:02:14.618153 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.618090    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-service-ca" (OuterVolumeSpecName: "service-ca") pod "ac19e41e-5d9b-4557-94c1-593730217095" (UID: "ac19e41e-5d9b-4557-94c1-593730217095"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:02:14.618365 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.618342    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ac19e41e-5d9b-4557-94c1-593730217095" (UID: "ac19e41e-5d9b-4557-94c1-593730217095"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:02:14.618431 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.618412    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-console-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:02:14.618482 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.618432    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:02:14.618482 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.618448    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-oauth-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:02:14.620099 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.620070    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ac19e41e-5d9b-4557-94c1-593730217095" (UID: "ac19e41e-5d9b-4557-94c1-593730217095"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:02:14.620219 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.620112    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ac19e41e-5d9b-4557-94c1-593730217095" (UID: "ac19e41e-5d9b-4557-94c1-593730217095"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:02:14.620219 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.620133    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac19e41e-5d9b-4557-94c1-593730217095-kube-api-access-fp7c7" (OuterVolumeSpecName: "kube-api-access-fp7c7") pod "ac19e41e-5d9b-4557-94c1-593730217095" (UID: "ac19e41e-5d9b-4557-94c1-593730217095"). InnerVolumeSpecName "kube-api-access-fp7c7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:02:14.719882 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.719839    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp7c7\" (UniqueName: \"kubernetes.io/projected/ac19e41e-5d9b-4557-94c1-593730217095-kube-api-access-fp7c7\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:02:14.719882 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.719874    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:02:14.719882 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.719891    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac19e41e-5d9b-4557-94c1-593730217095-console-oauth-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:02:14.720134 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:14.719904    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac19e41e-5d9b-4557-94c1-593730217095-trusted-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:02:15.305641 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:15.305609    2562 generic.go:356] "Generic (PLEG): container finished" podID="ac19e41e-5d9b-4557-94c1-593730217095" containerID="d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821" exitCode=0
Jul 23 16:02:15.306096 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:15.305665    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77ddd8c9d8-c6lqr"
Jul 23 16:02:15.306096 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:15.305691    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77ddd8c9d8-c6lqr" event={"ID":"ac19e41e-5d9b-4557-94c1-593730217095","Type":"ContainerDied","Data":"d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821"}
Jul 23 16:02:15.306096 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:15.305741    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77ddd8c9d8-c6lqr" event={"ID":"ac19e41e-5d9b-4557-94c1-593730217095","Type":"ContainerDied","Data":"a541afcd4eed067ed99ef3da7c9747589cd5b3f3f4612436c6a26c7005d9ce73"}
Jul 23 16:02:15.306096 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:15.305761    2562 scope.go:117] "RemoveContainer" containerID="d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821"
Jul 23 16:02:15.314752 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:15.314730    2562 scope.go:117] "RemoveContainer" containerID="d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821"
Jul 23 16:02:15.315056 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:02:15.315034    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821\": container with ID starting with d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821 not found: ID does not exist" containerID="d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821"
Jul 23 16:02:15.315120 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:15.315062    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821"} err="failed to get container status \"d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821\": rpc error: code = NotFound desc = could not find container \"d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821\": container with ID starting with d6ece01153408566aba4c2174f5800b965274b5c8e580c4e8f629e916366b821 not found: ID does not exist"
Jul 23 16:02:15.323715 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:15.323691    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77ddd8c9d8-c6lqr"]
Jul 23 16:02:15.328784 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:15.328762    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77ddd8c9d8-c6lqr"]
Jul 23 16:02:17.254914 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:02:17.254882    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac19e41e-5d9b-4557-94c1-593730217095" path="/var/lib/kubelet/pods/ac19e41e-5d9b-4557-94c1-593730217095/volumes"
Jul 23 16:03:57.167338 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:03:57.167265    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:03:57.167910 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:03:57.167893    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:04:00.789496 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.789463    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8498cdcf8c-rtngd"]
Jul 23 16:04:00.789912 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.789890    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac19e41e-5d9b-4557-94c1-593730217095" containerName="console"
Jul 23 16:04:00.789912 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.789904    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac19e41e-5d9b-4557-94c1-593730217095" containerName="console"
Jul 23 16:04:00.789997 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.789982    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac19e41e-5d9b-4557-94c1-593730217095" containerName="console"
Jul 23 16:04:00.793095 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.793079    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:00.797361 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.797340    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9p442\""
Jul 23 16:04:00.797516 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.797446    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 16:04:00.797516 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.797502    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 16:04:00.804683 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.804655    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8498cdcf8c-rtngd"]
Jul 23 16:04:00.820545 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.820511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn7xf\" (UniqueName: \"kubernetes.io/projected/27ca0618-69a1-4982-ab29-d1d043ec0d1f-kube-api-access-sn7xf\") pod \"maas-controller-8498cdcf8c-rtngd\" (UID: \"27ca0618-69a1-4982-ab29-d1d043ec0d1f\") " pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:00.820705 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.820597    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27ca0618-69a1-4982-ab29-d1d043ec0d1f-webhook-cert\") pod \"maas-controller-8498cdcf8c-rtngd\" (UID: \"27ca0618-69a1-4982-ab29-d1d043ec0d1f\") " pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:00.921316 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.921273    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn7xf\" (UniqueName: \"kubernetes.io/projected/27ca0618-69a1-4982-ab29-d1d043ec0d1f-kube-api-access-sn7xf\") pod \"maas-controller-8498cdcf8c-rtngd\" (UID: \"27ca0618-69a1-4982-ab29-d1d043ec0d1f\") " pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:00.921528 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.921434    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27ca0618-69a1-4982-ab29-d1d043ec0d1f-webhook-cert\") pod \"maas-controller-8498cdcf8c-rtngd\" (UID: \"27ca0618-69a1-4982-ab29-d1d043ec0d1f\") " pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:00.923882 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.923861    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27ca0618-69a1-4982-ab29-d1d043ec0d1f-webhook-cert\") pod \"maas-controller-8498cdcf8c-rtngd\" (UID: \"27ca0618-69a1-4982-ab29-d1d043ec0d1f\") " pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:00.930708 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:00.930671    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn7xf\" (UniqueName: \"kubernetes.io/projected/27ca0618-69a1-4982-ab29-d1d043ec0d1f-kube-api-access-sn7xf\") pod \"maas-controller-8498cdcf8c-rtngd\" (UID: \"27ca0618-69a1-4982-ab29-d1d043ec0d1f\") " pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:01.105295 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:01.105194    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:01.450584 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:01.450562    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8498cdcf8c-rtngd"]
Jul 23 16:04:01.452505 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:04:01.452480    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27ca0618_69a1_4982_ab29_d1d043ec0d1f.slice/crio-a7b1cbd95d4121304cb2d011e000c37c1be065acd4eb3f1e18f820fe63d9d1f0 WatchSource:0}: Error finding container a7b1cbd95d4121304cb2d011e000c37c1be065acd4eb3f1e18f820fe63d9d1f0: Status 404 returned error can't find the container with id a7b1cbd95d4121304cb2d011e000c37c1be065acd4eb3f1e18f820fe63d9d1f0
Jul 23 16:04:01.853167 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:01.853069    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8498cdcf8c-rtngd" event={"ID":"27ca0618-69a1-4982-ab29-d1d043ec0d1f","Type":"ContainerStarted","Data":"a7b1cbd95d4121304cb2d011e000c37c1be065acd4eb3f1e18f820fe63d9d1f0"}
Jul 23 16:04:04.872250 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:04.872211    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8498cdcf8c-rtngd" event={"ID":"27ca0618-69a1-4982-ab29-d1d043ec0d1f","Type":"ContainerStarted","Data":"c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9"}
Jul 23 16:04:04.872745 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:04.872271    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:04.893661 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:04.893608    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8498cdcf8c-rtngd" podStartSLOduration=2.4095074260000002 podStartE2EDuration="4.893590827s" totalImagesPullingTime="2.484083401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:04:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:04:04.892067912 +0000 UTC m=+908.096435317" watchObservedRunningTime="2026-07-23 16:04:04.893590827 +0000 UTC m=+908.097958220"
Jul 23 16:04:15.879087 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:15.879053    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:20.561674 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.561640    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8498cdcf8c-rtngd"]
Jul 23 16:04:20.562171 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.561858    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8498cdcf8c-rtngd" podUID="27ca0618-69a1-4982-ab29-d1d043ec0d1f" containerName="manager" containerID="cri-o://c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9" gracePeriod=30
Jul 23 16:04:20.814491 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.814422    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:20.906255 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.906220    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn7xf\" (UniqueName: \"kubernetes.io/projected/27ca0618-69a1-4982-ab29-d1d043ec0d1f-kube-api-access-sn7xf\") pod \"27ca0618-69a1-4982-ab29-d1d043ec0d1f\" (UID: \"27ca0618-69a1-4982-ab29-d1d043ec0d1f\") "
Jul 23 16:04:20.906447 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.906355    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27ca0618-69a1-4982-ab29-d1d043ec0d1f-webhook-cert\") pod \"27ca0618-69a1-4982-ab29-d1d043ec0d1f\" (UID: \"27ca0618-69a1-4982-ab29-d1d043ec0d1f\") "
Jul 23 16:04:20.908422 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.908391    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27ca0618-69a1-4982-ab29-d1d043ec0d1f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "27ca0618-69a1-4982-ab29-d1d043ec0d1f" (UID: "27ca0618-69a1-4982-ab29-d1d043ec0d1f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:04:20.908550 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.908391    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27ca0618-69a1-4982-ab29-d1d043ec0d1f-kube-api-access-sn7xf" (OuterVolumeSpecName: "kube-api-access-sn7xf") pod "27ca0618-69a1-4982-ab29-d1d043ec0d1f" (UID: "27ca0618-69a1-4982-ab29-d1d043ec0d1f"). InnerVolumeSpecName "kube-api-access-sn7xf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:04:20.952460 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.952427    2562 generic.go:356] "Generic (PLEG): container finished" podID="27ca0618-69a1-4982-ab29-d1d043ec0d1f" containerID="c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9" exitCode=0
Jul 23 16:04:20.952657 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.952467    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8498cdcf8c-rtngd" event={"ID":"27ca0618-69a1-4982-ab29-d1d043ec0d1f","Type":"ContainerDied","Data":"c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9"}
Jul 23 16:04:20.952657 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.952487    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8498cdcf8c-rtngd"
Jul 23 16:04:20.952657 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.952504    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8498cdcf8c-rtngd" event={"ID":"27ca0618-69a1-4982-ab29-d1d043ec0d1f","Type":"ContainerDied","Data":"a7b1cbd95d4121304cb2d011e000c37c1be065acd4eb3f1e18f820fe63d9d1f0"}
Jul 23 16:04:20.952657 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.952525    2562 scope.go:117] "RemoveContainer" containerID="c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9"
Jul 23 16:04:20.961745 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.961715    2562 scope.go:117] "RemoveContainer" containerID="c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9"
Jul 23 16:04:20.962092 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:04:20.962068    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9\": container with ID starting with c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9 not found: ID does not exist" containerID="c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9"
Jul 23 16:04:20.962193 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.962103    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9"} err="failed to get container status \"c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9\": rpc error: code = NotFound desc = could not find container \"c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9\": container with ID starting with c84e36b4d631f3b013f5cee9ce1e06fed4afbf18cfffcabeac89119afaf770f9 not found: ID does not exist"
Jul 23 16:04:20.979757 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.979727    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8498cdcf8c-rtngd"]
Jul 23 16:04:20.983646 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:20.983619    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8498cdcf8c-rtngd"]
Jul 23 16:04:21.007662 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.007635    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27ca0618-69a1-4982-ab29-d1d043ec0d1f-webhook-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:04:21.007662 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.007659    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn7xf\" (UniqueName: \"kubernetes.io/projected/27ca0618-69a1-4982-ab29-d1d043ec0d1f-kube-api-access-sn7xf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:04:21.254039 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.254007    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27ca0618-69a1-4982-ab29-d1d043ec0d1f" path="/var/lib/kubelet/pods/27ca0618-69a1-4982-ab29-d1d043ec0d1f/volumes"
Jul 23 16:04:21.819949 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.819913    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8498cdcf8c-dws5b"]
Jul 23 16:04:21.820337 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.820311    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27ca0618-69a1-4982-ab29-d1d043ec0d1f" containerName="manager"
Jul 23 16:04:21.820337 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.820321    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="27ca0618-69a1-4982-ab29-d1d043ec0d1f" containerName="manager"
Jul 23 16:04:21.820418 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.820386    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="27ca0618-69a1-4982-ab29-d1d043ec0d1f" containerName="manager"
Jul 23 16:04:21.825234 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.825210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8498cdcf8c-dws5b"
Jul 23 16:04:21.829278 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.829252    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 16:04:21.829436 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.829253    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 16:04:21.829436 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.829330    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9p442\""
Jul 23 16:04:21.835311 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.835281    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8498cdcf8c-dws5b"]
Jul 23 16:04:21.915792 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.915754    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8fj5\" (UniqueName: \"kubernetes.io/projected/0dbaf1a1-09be-4db8-92fe-d1a17ed4437e-kube-api-access-v8fj5\") pod \"maas-controller-8498cdcf8c-dws5b\" (UID: \"0dbaf1a1-09be-4db8-92fe-d1a17ed4437e\") " pod="opendatahub/maas-controller-8498cdcf8c-dws5b"
Jul 23 16:04:21.915996 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:21.915905    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dbaf1a1-09be-4db8-92fe-d1a17ed4437e-webhook-cert\") pod \"maas-controller-8498cdcf8c-dws5b\" (UID: \"0dbaf1a1-09be-4db8-92fe-d1a17ed4437e\") " pod="opendatahub/maas-controller-8498cdcf8c-dws5b"
Jul 23 16:04:22.017123 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:22.017082    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dbaf1a1-09be-4db8-92fe-d1a17ed4437e-webhook-cert\") pod \"maas-controller-8498cdcf8c-dws5b\" (UID: \"0dbaf1a1-09be-4db8-92fe-d1a17ed4437e\") " pod="opendatahub/maas-controller-8498cdcf8c-dws5b"
Jul 23 16:04:22.017328 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:22.017181    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8fj5\" (UniqueName: \"kubernetes.io/projected/0dbaf1a1-09be-4db8-92fe-d1a17ed4437e-kube-api-access-v8fj5\") pod \"maas-controller-8498cdcf8c-dws5b\" (UID: \"0dbaf1a1-09be-4db8-92fe-d1a17ed4437e\") " pod="opendatahub/maas-controller-8498cdcf8c-dws5b"
Jul 23 16:04:22.019513 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:22.019490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dbaf1a1-09be-4db8-92fe-d1a17ed4437e-webhook-cert\") pod \"maas-controller-8498cdcf8c-dws5b\" (UID: \"0dbaf1a1-09be-4db8-92fe-d1a17ed4437e\") " pod="opendatahub/maas-controller-8498cdcf8c-dws5b"
Jul 23 16:04:22.030061 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:22.030036    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8fj5\" (UniqueName: \"kubernetes.io/projected/0dbaf1a1-09be-4db8-92fe-d1a17ed4437e-kube-api-access-v8fj5\") pod \"maas-controller-8498cdcf8c-dws5b\" (UID: \"0dbaf1a1-09be-4db8-92fe-d1a17ed4437e\") " pod="opendatahub/maas-controller-8498cdcf8c-dws5b"
Jul 23 16:04:22.136328 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:22.136238    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8498cdcf8c-dws5b"
Jul 23 16:04:22.478549 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:22.478524    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8498cdcf8c-dws5b"]
Jul 23 16:04:22.480522 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:04:22.480492    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dbaf1a1_09be_4db8_92fe_d1a17ed4437e.slice/crio-c61ba324c144417ec66cad9bc3a2f804e40c4720882cab35afeb003176df9834 WatchSource:0}: Error finding container c61ba324c144417ec66cad9bc3a2f804e40c4720882cab35afeb003176df9834: Status 404 returned error can't find the container with id c61ba324c144417ec66cad9bc3a2f804e40c4720882cab35afeb003176df9834
Jul 23 16:04:22.966754 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:22.966719    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8498cdcf8c-dws5b" event={"ID":"0dbaf1a1-09be-4db8-92fe-d1a17ed4437e","Type":"ContainerStarted","Data":"c61ba324c144417ec66cad9bc3a2f804e40c4720882cab35afeb003176df9834"}
Jul 23 16:04:23.973426 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:23.973388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8498cdcf8c-dws5b" event={"ID":"0dbaf1a1-09be-4db8-92fe-d1a17ed4437e","Type":"ContainerStarted","Data":"1315eac02f25be6db2db8728f6235ce0ed81effdc880f6ac0e72703e980761ae"}
Jul 23 16:04:23.973888 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:23.973496    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8498cdcf8c-dws5b"
Jul 23 16:04:23.993941 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:23.993874    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8498cdcf8c-dws5b" podStartSLOduration=2.378599294 podStartE2EDuration="2.993853921s" totalImagesPullingTime="615.254627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:04:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:04:23.992955709 +0000 UTC m=+927.197323102" watchObservedRunningTime="2026-07-23 16:04:23.993853921 +0000 UTC m=+927.198221317"
Jul 23 16:04:34.980411 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:04:34.980381    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8498cdcf8c-dws5b"
Jul 23 16:08:57.198984 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:08:57.198894    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:08:57.201358 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:08:57.201338    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:09:13.701951 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:13.701907    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"]
Jul 23 16:09:13.702537 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:13.702168    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr" podUID="ad371a3c-60fa-4aa0-87cc-dea03c1b86f6" containerName="manager" containerID="cri-o://591ef56cfd91d8c42b873236e143e9fd5eb13087212d3490934fd791f11e589e" gracePeriod=10
Jul 23 16:09:16.465226 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:16.465194    2562 generic.go:356] "Generic (PLEG): container finished" podID="ad371a3c-60fa-4aa0-87cc-dea03c1b86f6" containerID="591ef56cfd91d8c42b873236e143e9fd5eb13087212d3490934fd791f11e589e" exitCode=0
Jul 23 16:09:16.465608 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:16.465259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr" event={"ID":"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6","Type":"ContainerDied","Data":"591ef56cfd91d8c42b873236e143e9fd5eb13087212d3490934fd791f11e589e"}
Jul 23 16:09:16.959488 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:16.959463    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 16:09:17.039228 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:17.039194    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-extensions-socket-volume\") pod \"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6\" (UID: \"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6\") "
Jul 23 16:09:17.039409 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:17.039332    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5tnw\" (UniqueName: \"kubernetes.io/projected/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-kube-api-access-h5tnw\") pod \"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6\" (UID: \"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6\") "
Jul 23 16:09:17.039566 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:17.039533    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ad371a3c-60fa-4aa0-87cc-dea03c1b86f6" (UID: "ad371a3c-60fa-4aa0-87cc-dea03c1b86f6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:09:17.039712 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:17.039676    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-extensions-socket-volume\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:09:17.041606 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:17.041581    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-kube-api-access-h5tnw" (OuterVolumeSpecName: "kube-api-access-h5tnw") pod "ad371a3c-60fa-4aa0-87cc-dea03c1b86f6" (UID: "ad371a3c-60fa-4aa0-87cc-dea03c1b86f6"). InnerVolumeSpecName "kube-api-access-h5tnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:09:17.141167 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:17.141077    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5tnw\" (UniqueName: \"kubernetes.io/projected/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6-kube-api-access-h5tnw\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:09:17.476348 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:17.476320    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"
Jul 23 16:09:17.476781 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:17.476346    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr" event={"ID":"ad371a3c-60fa-4aa0-87cc-dea03c1b86f6","Type":"ContainerDied","Data":"6ad09115e77f7948de7aae79ae1e4bba26a04f2781509136ad65a49b61f366f1"}
Jul 23 16:09:17.476781 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:17.476417    2562 scope.go:117] "RemoveContainer" containerID="591ef56cfd91d8c42b873236e143e9fd5eb13087212d3490934fd791f11e589e"
Jul 23 16:09:17.494997 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:17.494973    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"]
Jul 23 16:09:17.498427 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:17.498404    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hmrpr"]
Jul 23 16:09:19.255094 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:09:19.255061    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad371a3c-60fa-4aa0-87cc-dea03c1b86f6" path="/var/lib/kubelet/pods/ad371a3c-60fa-4aa0-87cc-dea03c1b86f6/volumes"
Jul 23 16:10:15.599582 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.599551    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"]
Jul 23 16:10:15.600061 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.599983    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad371a3c-60fa-4aa0-87cc-dea03c1b86f6" containerName="manager"
Jul 23 16:10:15.600061 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.599996    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad371a3c-60fa-4aa0-87cc-dea03c1b86f6" containerName="manager"
Jul 23 16:10:15.600139 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.600073    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad371a3c-60fa-4aa0-87cc-dea03c1b86f6" containerName="manager"
Jul 23 16:10:15.604598 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.604577    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"
Jul 23 16:10:15.607457 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.607439    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-m2dnw\""
Jul 23 16:10:15.615445 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.615421    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"]
Jul 23 16:10:15.697316 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.697282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw6bt\" (UniqueName: \"kubernetes.io/projected/393a52f5-014c-4c5c-b1e7-654aa169a5da-kube-api-access-mw6bt\") pod \"kuadrant-operator-controller-manager-86878c769f-p799l\" (UID: \"393a52f5-014c-4c5c-b1e7-654aa169a5da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"
Jul 23 16:10:15.697512 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.697357    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/393a52f5-014c-4c5c-b1e7-654aa169a5da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p799l\" (UID: \"393a52f5-014c-4c5c-b1e7-654aa169a5da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"
Jul 23 16:10:15.798787 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.798741    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw6bt\" (UniqueName: \"kubernetes.io/projected/393a52f5-014c-4c5c-b1e7-654aa169a5da-kube-api-access-mw6bt\") pod \"kuadrant-operator-controller-manager-86878c769f-p799l\" (UID: \"393a52f5-014c-4c5c-b1e7-654aa169a5da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"
Jul 23 16:10:15.799002 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.798849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/393a52f5-014c-4c5c-b1e7-654aa169a5da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p799l\" (UID: \"393a52f5-014c-4c5c-b1e7-654aa169a5da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"
Jul 23 16:10:15.799183 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.799167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/393a52f5-014c-4c5c-b1e7-654aa169a5da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p799l\" (UID: \"393a52f5-014c-4c5c-b1e7-654aa169a5da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"
Jul 23 16:10:15.812039 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.812008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw6bt\" (UniqueName: \"kubernetes.io/projected/393a52f5-014c-4c5c-b1e7-654aa169a5da-kube-api-access-mw6bt\") pod \"kuadrant-operator-controller-manager-86878c769f-p799l\" (UID: \"393a52f5-014c-4c5c-b1e7-654aa169a5da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"
Jul 23 16:10:15.915649 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:15.915552    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"
Jul 23 16:10:16.059689 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:16.059663    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"]
Jul 23 16:10:16.062316 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:10:16.062282    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod393a52f5_014c_4c5c_b1e7_654aa169a5da.slice/crio-9dc4ed41b509a33befefb0d259a295a70290250d7465898afb76d54d6ca1dd58 WatchSource:0}: Error finding container 9dc4ed41b509a33befefb0d259a295a70290250d7465898afb76d54d6ca1dd58: Status 404 returned error can't find the container with id 9dc4ed41b509a33befefb0d259a295a70290250d7465898afb76d54d6ca1dd58
Jul 23 16:10:16.775898 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:16.775866    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l" event={"ID":"393a52f5-014c-4c5c-b1e7-654aa169a5da","Type":"ContainerStarted","Data":"09232698f9e6172b1da411402249fb14afaf1b957978398837daad2f0747b76a"}
Jul 23 16:10:16.776308 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:16.775917    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l" event={"ID":"393a52f5-014c-4c5c-b1e7-654aa169a5da","Type":"ContainerStarted","Data":"9dc4ed41b509a33befefb0d259a295a70290250d7465898afb76d54d6ca1dd58"}
Jul 23 16:10:16.776308 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:16.776019    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"
Jul 23 16:10:16.820197 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:16.820133    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l" podStartSLOduration=1.8201120020000001 podStartE2EDuration="1.820112002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:10:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:10:16.818760804 +0000 UTC m=+1280.023128199" watchObservedRunningTime="2026-07-23 16:10:16.820112002 +0000 UTC m=+1280.024479395"
Jul 23 16:10:27.783694 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:10:27.783659    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p799l"
Jul 23 16:12:55.117512 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.117417    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"]
Jul 23 16:12:55.121876 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.121848    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.124827 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.124782    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-bbbeb353-kserve-self-signed-certs\""
Jul 23 16:12:55.131273 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.131247    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"]
Jul 23 16:12:55.203033 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.202999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-dshm\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.203033 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.203038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tls-certs\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.203248 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.203082    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-model-cache\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.203248 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.203179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tmp-dir\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.203248 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.203224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42nfr\" (UniqueName: \"kubernetes.io/projected/843d6cc3-7eee-4d44-af39-3d5208c28ef2-kube-api-access-42nfr\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.203352 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.203274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-home\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.304399 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.304359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tls-certs\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.304592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.304416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-model-cache\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.304592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.304451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tmp-dir\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.304592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.304483    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42nfr\" (UniqueName: \"kubernetes.io/projected/843d6cc3-7eee-4d44-af39-3d5208c28ef2-kube-api-access-42nfr\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.304592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.304510    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-home\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.304592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.304577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-dshm\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.304999 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.304976    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-home\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.305079 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.305005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-model-cache\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.305079 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.305024    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tmp-dir\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.306687 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.306670    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-dshm\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.306971 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.306954    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tls-certs\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.338498 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.338472    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42nfr\" (UniqueName: \"kubernetes.io/projected/843d6cc3-7eee-4d44-af39-3d5208c28ef2-kube-api-access-42nfr\") pod \"e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") " pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.432623 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.432583    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:12:55.573228 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.573203    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"]
Jul 23 16:12:55.575316 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:12:55.575288    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod843d6cc3_7eee_4d44_af39_3d5208c28ef2.slice/crio-1f1ea7ac123b4f96788ebdc672f869ad022d181387e6b6525cb94db341127ab0 WatchSource:0}: Error finding container 1f1ea7ac123b4f96788ebdc672f869ad022d181387e6b6525cb94db341127ab0: Status 404 returned error can't find the container with id 1f1ea7ac123b4f96788ebdc672f869ad022d181387e6b6525cb94db341127ab0
Jul 23 16:12:55.590876 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:55.590846    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx" event={"ID":"843d6cc3-7eee-4d44-af39-3d5208c28ef2","Type":"ContainerStarted","Data":"1f1ea7ac123b4f96788ebdc672f869ad022d181387e6b6525cb94db341127ab0"}
Jul 23 16:12:56.598032 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:56.597999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx" event={"ID":"843d6cc3-7eee-4d44-af39-3d5208c28ef2","Type":"ContainerStarted","Data":"6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318"}
Jul 23 16:12:56.616735 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:12:56.616674    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx" podStartSLOduration=1.616658519 podStartE2EDuration="1.616658519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:12:56.61587297 +0000 UTC m=+1439.820240379" watchObservedRunningTime="2026-07-23 16:12:56.616658519 +0000 UTC m=+1439.821025911"
Jul 23 16:13:05.433128 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:05.433089    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:13:05.433128 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:05.433140    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:13:05.445598 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:05.445573    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:13:05.655576 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:05.655548    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:13:35.921092 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:35.921057    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"]
Jul 23 16:13:50.942746 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:50.942685    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx" podUID="843d6cc3-7eee-4d44-af39-3d5208c28ef2" containerName="main" containerID="cri-o://6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318" gracePeriod=45
Jul 23 16:13:51.208556 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.208483    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:13:51.366647 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.366611    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-dshm\") pod \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") "
Jul 23 16:13:51.366883 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.366702    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-model-cache\") pod \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") "
Jul 23 16:13:51.366883 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.366778    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-home\") pod \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") "
Jul 23 16:13:51.366883 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.366827    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tmp-dir\") pod \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") "
Jul 23 16:13:51.366883 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.366866    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42nfr\" (UniqueName: \"kubernetes.io/projected/843d6cc3-7eee-4d44-af39-3d5208c28ef2-kube-api-access-42nfr\") pod \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") "
Jul 23 16:13:51.367102 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.366897    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tls-certs\") pod \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\" (UID: \"843d6cc3-7eee-4d44-af39-3d5208c28ef2\") "
Jul 23 16:13:51.367102 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.367013    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-model-cache" (OuterVolumeSpecName: "model-cache") pod "843d6cc3-7eee-4d44-af39-3d5208c28ef2" (UID: "843d6cc3-7eee-4d44-af39-3d5208c28ef2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:13:51.367102 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.367067    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-home" (OuterVolumeSpecName: "home") pod "843d6cc3-7eee-4d44-af39-3d5208c28ef2" (UID: "843d6cc3-7eee-4d44-af39-3d5208c28ef2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:13:51.367259 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.367097    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "843d6cc3-7eee-4d44-af39-3d5208c28ef2" (UID: "843d6cc3-7eee-4d44-af39-3d5208c28ef2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:13:51.367259 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.367251    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-home\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:13:51.367370 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.367272    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tmp-dir\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:13:51.367370 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.367288    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-model-cache\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:13:51.369001 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.368957    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-dshm" (OuterVolumeSpecName: "dshm") pod "843d6cc3-7eee-4d44-af39-3d5208c28ef2" (UID: "843d6cc3-7eee-4d44-af39-3d5208c28ef2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:13:51.369130 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.369033    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "843d6cc3-7eee-4d44-af39-3d5208c28ef2" (UID: "843d6cc3-7eee-4d44-af39-3d5208c28ef2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:13:51.369130 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.369043    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/843d6cc3-7eee-4d44-af39-3d5208c28ef2-kube-api-access-42nfr" (OuterVolumeSpecName: "kube-api-access-42nfr") pod "843d6cc3-7eee-4d44-af39-3d5208c28ef2" (UID: "843d6cc3-7eee-4d44-af39-3d5208c28ef2"). InnerVolumeSpecName "kube-api-access-42nfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:13:51.468780 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.468682    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843d6cc3-7eee-4d44-af39-3d5208c28ef2-dshm\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:13:51.468780 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.468713    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42nfr\" (UniqueName: \"kubernetes.io/projected/843d6cc3-7eee-4d44-af39-3d5208c28ef2-kube-api-access-42nfr\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:13:51.468780 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.468732    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843d6cc3-7eee-4d44-af39-3d5208c28ef2-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:13:51.884863 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.884748    2562 generic.go:356] "Generic (PLEG): container finished" podID="843d6cc3-7eee-4d44-af39-3d5208c28ef2" containerID="6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318" exitCode=0
Jul 23 16:13:51.884863 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.884788    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx" event={"ID":"843d6cc3-7eee-4d44-af39-3d5208c28ef2","Type":"ContainerDied","Data":"6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318"}
Jul 23 16:13:51.884863 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.884818    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"
Jul 23 16:13:51.884863 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.884843    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx" event={"ID":"843d6cc3-7eee-4d44-af39-3d5208c28ef2","Type":"ContainerDied","Data":"1f1ea7ac123b4f96788ebdc672f869ad022d181387e6b6525cb94db341127ab0"}
Jul 23 16:13:51.884863 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.884865    2562 scope.go:117] "RemoveContainer" containerID="6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318"
Jul 23 16:13:51.894063 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.894045    2562 scope.go:117] "RemoveContainer" containerID="6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318"
Jul 23 16:13:51.894339 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:13:51.894319    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318\": container with ID starting with 6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318 not found: ID does not exist" containerID="6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318"
Jul 23 16:13:51.894388 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.894349    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318"} err="failed to get container status \"6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318\": rpc error: code = NotFound desc = could not find container \"6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318\": container with ID starting with 6fd5f594d5c3ce202b0053b5ccdb6c201aced4e96c1d1fd63fc0c4dd1c614318 not found: ID does not exist"
Jul 23 16:13:51.907730 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.907702    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"]
Jul 23 16:13:51.911596 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:51.911531    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-bbbeb353-kserve-9f8bd6fd7-vhkxx"]
Jul 23 16:13:53.253948 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:53.253913    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="843d6cc3-7eee-4d44-af39-3d5208c28ef2" path="/var/lib/kubelet/pods/843d6cc3-7eee-4d44-af39-3d5208c28ef2/volumes"
Jul 23 16:13:57.234697 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:57.234668    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:13:57.236257 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:13:57.236229    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:15:00.142936 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:00.142903    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6"]
Jul 23 16:15:00.145323 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:00.143304    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843d6cc3-7eee-4d44-af39-3d5208c28ef2" containerName="main"
Jul 23 16:15:00.145323 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:00.143314    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="843d6cc3-7eee-4d44-af39-3d5208c28ef2" containerName="main"
Jul 23 16:15:00.145323 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:00.143385    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="843d6cc3-7eee-4d44-af39-3d5208c28ef2" containerName="main"
Jul 23 16:15:00.146260 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:00.146244    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6"
Jul 23 16:15:00.149448 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:00.149430    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w6jwk\""
Jul 23 16:15:00.153376 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:00.153348    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6"]
Jul 23 16:15:00.158507 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:00.158483    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6"
Jul 23 16:15:00.298487 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:00.298458    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6"]
Jul 23 16:15:00.301493 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:15:00.301468    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab27902_ab28_4ce4_adb9_1ab7cb149f22.slice/crio-833d49e951b959bbf166800cd726924e5d7e6e422b8664d3cc53ba9a0cc20c05 WatchSource:0}: Error finding container 833d49e951b959bbf166800cd726924e5d7e6e422b8664d3cc53ba9a0cc20c05: Status 404 returned error can't find the container with id 833d49e951b959bbf166800cd726924e5d7e6e422b8664d3cc53ba9a0cc20c05
Jul 23 16:15:01.255898 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:01.255852    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6" event={"ID":"7ab27902-ab28-4ce4-adb9-1ab7cb149f22","Type":"ContainerStarted","Data":"55b569a7336f1b358c24a29a5c14d3bb53c2a2da557e0d8ce06299abc37a2924"}
Jul 23 16:15:01.256406 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:01.255919    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6" event={"ID":"7ab27902-ab28-4ce4-adb9-1ab7cb149f22","Type":"ContainerStarted","Data":"833d49e951b959bbf166800cd726924e5d7e6e422b8664d3cc53ba9a0cc20c05"}
Jul 23 16:15:01.269857 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:01.269781    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6" podStartSLOduration=1.26976308 podStartE2EDuration="1.26976308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:15:01.267660807 +0000 UTC m=+1564.472028201" watchObservedRunningTime="2026-07-23 16:15:01.26976308 +0000 UTC m=+1564.474130472"
Jul 23 16:15:02.257457 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:02.257426    2562 generic.go:356] "Generic (PLEG): container finished" podID="7ab27902-ab28-4ce4-adb9-1ab7cb149f22" containerID="55b569a7336f1b358c24a29a5c14d3bb53c2a2da557e0d8ce06299abc37a2924" exitCode=0
Jul 23 16:15:02.258003 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:02.257493    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6" event={"ID":"7ab27902-ab28-4ce4-adb9-1ab7cb149f22","Type":"ContainerDied","Data":"55b569a7336f1b358c24a29a5c14d3bb53c2a2da557e0d8ce06299abc37a2924"}
Jul 23 16:15:03.399576 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:03.399550    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6"
Jul 23 16:15:04.269079 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:04.269045    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6"
Jul 23 16:15:04.269254 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:04.269049    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747055-9lfv6" event={"ID":"7ab27902-ab28-4ce4-adb9-1ab7cb149f22","Type":"ContainerDied","Data":"833d49e951b959bbf166800cd726924e5d7e6e422b8664d3cc53ba9a0cc20c05"}
Jul 23 16:15:04.269254 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:15:04.269177    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="833d49e951b959bbf166800cd726924e5d7e6e422b8664d3cc53ba9a0cc20c05"
Jul 23 16:17:27.619262 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.619177    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"]
Jul 23 16:17:27.620117 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.619776    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab27902-ab28-4ce4-adb9-1ab7cb149f22" containerName="cleanup"
Jul 23 16:17:27.620117 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.619795    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab27902-ab28-4ce4-adb9-1ab7cb149f22" containerName="cleanup"
Jul 23 16:17:27.620117 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.619948    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ab27902-ab28-4ce4-adb9-1ab7cb149f22" containerName="cleanup"
Jul 23 16:17:27.623381 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.623359    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:27.626140 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.626118    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5j8mj\""
Jul 23 16:17:27.626243 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.626205    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8e175082-serving-cert\""
Jul 23 16:17:27.627400 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.627376    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 16:17:27.627400 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.627394    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 16:17:27.634113 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.634084    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"]
Jul 23 16:17:27.660748 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.660716    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081781fd-75cd-4066-af42-613c64e75223-openshift-service-ca\") pod \"maas-api-e2e-ait-8e175082-58476c7df-pc5n5\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:27.660963 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.660845    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd7gc\" (UniqueName: \"kubernetes.io/projected/081781fd-75cd-4066-af42-613c64e75223-kube-api-access-gd7gc\") pod \"maas-api-e2e-ait-8e175082-58476c7df-pc5n5\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:27.660963 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.660906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081781fd-75cd-4066-af42-613c64e75223-maas-api-tls\") pod \"maas-api-e2e-ait-8e175082-58476c7df-pc5n5\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:27.762281 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.762243    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081781fd-75cd-4066-af42-613c64e75223-openshift-service-ca\") pod \"maas-api-e2e-ait-8e175082-58476c7df-pc5n5\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:27.762475 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.762384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd7gc\" (UniqueName: \"kubernetes.io/projected/081781fd-75cd-4066-af42-613c64e75223-kube-api-access-gd7gc\") pod \"maas-api-e2e-ait-8e175082-58476c7df-pc5n5\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:27.762475 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.762437    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081781fd-75cd-4066-af42-613c64e75223-maas-api-tls\") pod \"maas-api-e2e-ait-8e175082-58476c7df-pc5n5\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:27.762606 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:17:27.762561    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-serving-cert: secret "maas-api-e2e-ait-8e175082-serving-cert" not found
Jul 23 16:17:27.762661 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:17:27.762640    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/081781fd-75cd-4066-af42-613c64e75223-maas-api-tls podName:081781fd-75cd-4066-af42-613c64e75223 nodeName:}" failed. No retries permitted until 2026-07-23 16:17:28.2626185 +0000 UTC m=+1711.466985874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/081781fd-75cd-4066-af42-613c64e75223-maas-api-tls") pod "maas-api-e2e-ait-8e175082-58476c7df-pc5n5" (UID: "081781fd-75cd-4066-af42-613c64e75223") : secret "maas-api-e2e-ait-8e175082-serving-cert" not found
Jul 23 16:17:27.762946 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.762925    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081781fd-75cd-4066-af42-613c64e75223-openshift-service-ca\") pod \"maas-api-e2e-ait-8e175082-58476c7df-pc5n5\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:27.771727 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:27.771689    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd7gc\" (UniqueName: \"kubernetes.io/projected/081781fd-75cd-4066-af42-613c64e75223-kube-api-access-gd7gc\") pod \"maas-api-e2e-ait-8e175082-58476c7df-pc5n5\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:28.268471 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:28.268436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081781fd-75cd-4066-af42-613c64e75223-maas-api-tls\") pod \"maas-api-e2e-ait-8e175082-58476c7df-pc5n5\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:28.270938 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:28.270912    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081781fd-75cd-4066-af42-613c64e75223-maas-api-tls\") pod \"maas-api-e2e-ait-8e175082-58476c7df-pc5n5\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:28.534976 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:28.534870    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:28.683169 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:28.683129    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"]
Jul 23 16:17:28.685568 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:17:28.685542    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081781fd_75cd_4066_af42_613c64e75223.slice/crio-7fba9ed018f286a795bd0198bcdbae948196880140f7721caab84c0c7f7f5551 WatchSource:0}: Error finding container 7fba9ed018f286a795bd0198bcdbae948196880140f7721caab84c0c7f7f5551: Status 404 returned error can't find the container with id 7fba9ed018f286a795bd0198bcdbae948196880140f7721caab84c0c7f7f5551
Jul 23 16:17:28.686978 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:28.686960    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 16:17:29.021454 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:29.021419    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5" event={"ID":"081781fd-75cd-4066-af42-613c64e75223","Type":"ContainerStarted","Data":"7fba9ed018f286a795bd0198bcdbae948196880140f7721caab84c0c7f7f5551"}
Jul 23 16:17:31.033882 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:31.033839    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5" event={"ID":"081781fd-75cd-4066-af42-613c64e75223","Type":"ContainerStarted","Data":"568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1"}
Jul 23 16:17:31.034276 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:31.033903    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:31.051855 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:31.051781    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5" podStartSLOduration=2.181921084 podStartE2EDuration="4.051762503s" totalImagesPullingTime="1.869841419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:17:31.05084203 +0000 UTC m=+1714.255209423" watchObservedRunningTime="2026-07-23 16:17:31.051762503 +0000 UTC m=+1714.256129896"
Jul 23 16:17:37.044662 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:37.044632    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:17:53.164544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:53.164503    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"]
Jul 23 16:17:53.168199 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:53.168177    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"
Jul 23 16:17:53.170979 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:53.170957    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w6jwk\""
Jul 23 16:17:53.177233 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:53.176961    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"]
Jul 23 16:17:53.211045 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:53.211006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/404179e4-2db0-4c58-a15e-b9d6579b950f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8e175082-bmtwt\" (UID: \"404179e4-2db0-4c58-a15e-b9d6579b950f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"
Jul 23 16:17:53.311816 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:53.311775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/404179e4-2db0-4c58-a15e-b9d6579b950f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8e175082-bmtwt\" (UID: \"404179e4-2db0-4c58-a15e-b9d6579b950f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"
Jul 23 16:17:53.312415 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:53.312390    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/404179e4-2db0-4c58-a15e-b9d6579b950f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8e175082-bmtwt\" (UID: \"404179e4-2db0-4c58-a15e-b9d6579b950f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"
Jul 23 16:17:53.481535 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:53.481500    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"
Jul 23 16:17:53.618794 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:53.618772    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"]
Jul 23 16:17:53.621225 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:17:53.621197    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod404179e4_2db0_4c58_a15e_b9d6579b950f.slice/crio-72021a4ebd4dac6456f0fc19ff6467decc9f0e8e67b974affa587498db3ca28c WatchSource:0}: Error finding container 72021a4ebd4dac6456f0fc19ff6467decc9f0e8e67b974affa587498db3ca28c: Status 404 returned error can't find the container with id 72021a4ebd4dac6456f0fc19ff6467decc9f0e8e67b974affa587498db3ca28c
Jul 23 16:17:54.169044 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:54.169015    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt" event={"ID":"404179e4-2db0-4c58-a15e-b9d6579b950f","Type":"ContainerStarted","Data":"56f4c1328420f7814e9d3fd1797d36592759b8b2e8c87fb6b213fbe191d6614f"}
Jul 23 16:17:54.169494 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:54.169070    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt" event={"ID":"404179e4-2db0-4c58-a15e-b9d6579b950f","Type":"ContainerStarted","Data":"72021a4ebd4dac6456f0fc19ff6467decc9f0e8e67b974affa587498db3ca28c"}
Jul 23 16:17:54.209957 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:54.209906    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt" podStartSLOduration=1.209889318 podStartE2EDuration="1.209889318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:17:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:17:54.205420492 +0000 UTC m=+1737.409787885" watchObservedRunningTime="2026-07-23 16:17:54.209889318 +0000 UTC m=+1737.414256710"
Jul 23 16:17:55.176396 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:55.176363    2562 generic.go:356] "Generic (PLEG): container finished" podID="404179e4-2db0-4c58-a15e-b9d6579b950f" containerID="56f4c1328420f7814e9d3fd1797d36592759b8b2e8c87fb6b213fbe191d6614f" exitCode=0
Jul 23 16:17:55.176867 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:55.176413    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt" event={"ID":"404179e4-2db0-4c58-a15e-b9d6579b950f","Type":"ContainerDied","Data":"56f4c1328420f7814e9d3fd1797d36592759b8b2e8c87fb6b213fbe191d6614f"}
Jul 23 16:17:56.316117 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:56.316093    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"
Jul 23 16:17:56.341129 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:56.341101    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/404179e4-2db0-4c58-a15e-b9d6579b950f-maas-api-service-ca\") pod \"404179e4-2db0-4c58-a15e-b9d6579b950f\" (UID: \"404179e4-2db0-4c58-a15e-b9d6579b950f\") "
Jul 23 16:17:56.341473 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:56.341451    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/404179e4-2db0-4c58-a15e-b9d6579b950f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "404179e4-2db0-4c58-a15e-b9d6579b950f" (UID: "404179e4-2db0-4c58-a15e-b9d6579b950f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:17:56.341564 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:56.341547    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/404179e4-2db0-4c58-a15e-b9d6579b950f-maas-api-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:17:57.187572 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:57.187542    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"
Jul 23 16:17:57.187745 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:57.187546    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt" event={"ID":"404179e4-2db0-4c58-a15e-b9d6579b950f","Type":"ContainerDied","Data":"72021a4ebd4dac6456f0fc19ff6467decc9f0e8e67b974affa587498db3ca28c"}
Jul 23 16:17:57.187745 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:17:57.187665    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72021a4ebd4dac6456f0fc19ff6467decc9f0e8e67b974affa587498db3ca28c"
Jul 23 16:18:03.733173 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:03.733136    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"]
Jul 23 16:18:03.733679 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:03.733356    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5" podUID="081781fd-75cd-4066-af42-613c64e75223" containerName="maas-api" containerID="cri-o://568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1" gracePeriod=30
Jul 23 16:18:03.986181 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:03.986118    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:18:04.115524 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.115492    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081781fd-75cd-4066-af42-613c64e75223-maas-api-tls\") pod \"081781fd-75cd-4066-af42-613c64e75223\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") "
Jul 23 16:18:04.115731 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.115538    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081781fd-75cd-4066-af42-613c64e75223-openshift-service-ca\") pod \"081781fd-75cd-4066-af42-613c64e75223\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") "
Jul 23 16:18:04.115731 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.115631    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd7gc\" (UniqueName: \"kubernetes.io/projected/081781fd-75cd-4066-af42-613c64e75223-kube-api-access-gd7gc\") pod \"081781fd-75cd-4066-af42-613c64e75223\" (UID: \"081781fd-75cd-4066-af42-613c64e75223\") "
Jul 23 16:18:04.116022 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.115998    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/081781fd-75cd-4066-af42-613c64e75223-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "081781fd-75cd-4066-af42-613c64e75223" (UID: "081781fd-75cd-4066-af42-613c64e75223"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:18:04.118001 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.117977    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/081781fd-75cd-4066-af42-613c64e75223-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "081781fd-75cd-4066-af42-613c64e75223" (UID: "081781fd-75cd-4066-af42-613c64e75223"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:18:04.118105 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.118084    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081781fd-75cd-4066-af42-613c64e75223-kube-api-access-gd7gc" (OuterVolumeSpecName: "kube-api-access-gd7gc") pod "081781fd-75cd-4066-af42-613c64e75223" (UID: "081781fd-75cd-4066-af42-613c64e75223"). InnerVolumeSpecName "kube-api-access-gd7gc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:18:04.216755 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.216708    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081781fd-75cd-4066-af42-613c64e75223-maas-api-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:18:04.216755 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.216748    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081781fd-75cd-4066-af42-613c64e75223-openshift-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:18:04.216755 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.216760    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd7gc\" (UniqueName: \"kubernetes.io/projected/081781fd-75cd-4066-af42-613c64e75223-kube-api-access-gd7gc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:18:04.226089 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.226060    2562 generic.go:356] "Generic (PLEG): container finished" podID="081781fd-75cd-4066-af42-613c64e75223" containerID="568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1" exitCode=0
Jul 23 16:18:04.226232 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.226098    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5" event={"ID":"081781fd-75cd-4066-af42-613c64e75223","Type":"ContainerDied","Data":"568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1"}
Jul 23 16:18:04.226232 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.226106    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"
Jul 23 16:18:04.226232 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.226141    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5" event={"ID":"081781fd-75cd-4066-af42-613c64e75223","Type":"ContainerDied","Data":"7fba9ed018f286a795bd0198bcdbae948196880140f7721caab84c0c7f7f5551"}
Jul 23 16:18:04.226232 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.226167    2562 scope.go:117] "RemoveContainer" containerID="568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1"
Jul 23 16:18:04.235429 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.235408    2562 scope.go:117] "RemoveContainer" containerID="568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1"
Jul 23 16:18:04.235692 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:18:04.235673    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1\": container with ID starting with 568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1 not found: ID does not exist" containerID="568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1"
Jul 23 16:18:04.235758 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.235705    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1"} err="failed to get container status \"568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1\": rpc error: code = NotFound desc = could not find container \"568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1\": container with ID starting with 568d14edef3f3c0cef7e5e9c43923114c7d8ec83207fb0b6f9ecbd090fa51ef1 not found: ID does not exist"
Jul 23 16:18:04.247849 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.247774    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"]
Jul 23 16:18:04.251271 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:04.251250    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8e175082-58476c7df-pc5n5"]
Jul 23 16:18:05.254380 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:05.254346    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="081781fd-75cd-4066-af42-613c64e75223" path="/var/lib/kubelet/pods/081781fd-75cd-4066-af42-613c64e75223/volumes"
Jul 23 16:18:08.247185 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:08.247152    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"]
Jul 23 16:18:08.250664 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:08.250638    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8e175082-bmtwt"]
Jul 23 16:18:09.255053 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:09.255015    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="404179e4-2db0-4c58-a15e-b9d6579b950f" path="/var/lib/kubelet/pods/404179e4-2db0-4c58-a15e-b9d6579b950f/volumes"
Jul 23 16:18:12.932936 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.932885    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"]
Jul 23 16:18:12.933590 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.933570    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="081781fd-75cd-4066-af42-613c64e75223" containerName="maas-api"
Jul 23 16:18:12.933590 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.933592    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="081781fd-75cd-4066-af42-613c64e75223" containerName="maas-api"
Jul 23 16:18:12.933752 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.933609    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="404179e4-2db0-4c58-a15e-b9d6579b950f" containerName="revoke-keys"
Jul 23 16:18:12.933752 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.933618    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="404179e4-2db0-4c58-a15e-b9d6579b950f" containerName="revoke-keys"
Jul 23 16:18:12.933752 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.933712    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="081781fd-75cd-4066-af42-613c64e75223" containerName="maas-api"
Jul 23 16:18:12.933752 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.933730    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="404179e4-2db0-4c58-a15e-b9d6579b950f" containerName="revoke-keys"
Jul 23 16:18:12.938592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.938572    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:12.941843 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.941821    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5j8mj\""
Jul 23 16:18:12.942248 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.942231    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 16:18:12.943062 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.943043    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 16:18:12.943384 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.943365    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-8f4ec811-serving-cert\""
Jul 23 16:18:12.951850 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.951825    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"]
Jul 23 16:18:12.997689 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.997654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87da44a5-ced4-466f-abb8-165c3f397932-maas-api-tls\") pod \"maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:12.997897 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.997704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prf2m\" (UniqueName: \"kubernetes.io/projected/87da44a5-ced4-466f-abb8-165c3f397932-kube-api-access-prf2m\") pod \"maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:12.997897 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:12.997737    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87da44a5-ced4-466f-abb8-165c3f397932-openshift-service-ca\") pod \"maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:13.098467 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:13.098423    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87da44a5-ced4-466f-abb8-165c3f397932-maas-api-tls\") pod \"maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:13.098656 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:13.098489    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prf2m\" (UniqueName: \"kubernetes.io/projected/87da44a5-ced4-466f-abb8-165c3f397932-kube-api-access-prf2m\") pod \"maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:13.098656 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:13.098535    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87da44a5-ced4-466f-abb8-165c3f397932-openshift-service-ca\") pod \"maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:13.099286 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:13.099264    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87da44a5-ced4-466f-abb8-165c3f397932-openshift-service-ca\") pod \"maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:13.100980 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:13.100961    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87da44a5-ced4-466f-abb8-165c3f397932-maas-api-tls\") pod \"maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:13.107396 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:13.107374    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prf2m\" (UniqueName: \"kubernetes.io/projected/87da44a5-ced4-466f-abb8-165c3f397932-kube-api-access-prf2m\") pod \"maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:13.250284 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:13.250190    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:13.398222 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:13.398196    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"]
Jul 23 16:18:13.400523 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:18:13.400490    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87da44a5_ced4_466f_abb8_165c3f397932.slice/crio-b3560a5c2ca9040a3d983e32b286e9109a0c077fdf53f9e4eba53e56107f9bb0 WatchSource:0}: Error finding container b3560a5c2ca9040a3d983e32b286e9109a0c077fdf53f9e4eba53e56107f9bb0: Status 404 returned error can't find the container with id b3560a5c2ca9040a3d983e32b286e9109a0c077fdf53f9e4eba53e56107f9bb0
Jul 23 16:18:14.284319 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:14.284284    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc" event={"ID":"87da44a5-ced4-466f-abb8-165c3f397932","Type":"ContainerStarted","Data":"b3560a5c2ca9040a3d983e32b286e9109a0c077fdf53f9e4eba53e56107f9bb0"}
Jul 23 16:18:15.291109 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:15.291075    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc" event={"ID":"87da44a5-ced4-466f-abb8-165c3f397932","Type":"ContainerStarted","Data":"e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0"}
Jul 23 16:18:15.291526 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:15.291198    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:15.308658 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:15.308603    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc" podStartSLOduration=2.148152127 podStartE2EDuration="3.308585093s" totalImagesPullingTime="1.160432966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:18:15.307381314 +0000 UTC m=+1758.511748733" watchObservedRunningTime="2026-07-23 16:18:15.308585093 +0000 UTC m=+1758.512952494"
Jul 23 16:18:21.302501 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:21.302466    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:37.987330 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:37.987291    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"]
Jul 23 16:18:37.990852 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:37.990835    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"
Jul 23 16:18:37.993579 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:37.993555    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w6jwk\""
Jul 23 16:18:37.997776 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:37.997747    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"]
Jul 23 16:18:38.031404 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:38.031360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd87dec5-f7e2-43a4-a2db-d5428f800f09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx\" (UID: \"cd87dec5-f7e2-43a4-a2db-d5428f800f09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"
Jul 23 16:18:38.132101 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:38.132067    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd87dec5-f7e2-43a4-a2db-d5428f800f09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx\" (UID: \"cd87dec5-f7e2-43a4-a2db-d5428f800f09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"
Jul 23 16:18:38.132746 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:38.132721    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd87dec5-f7e2-43a4-a2db-d5428f800f09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx\" (UID: \"cd87dec5-f7e2-43a4-a2db-d5428f800f09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"
Jul 23 16:18:38.303699 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:38.303610    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"
Jul 23 16:18:38.460769 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:38.460737    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"]
Jul 23 16:18:38.462833 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:18:38.462792    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd87dec5_f7e2_43a4_a2db_d5428f800f09.slice/crio-e6098c957732b30cbb41c5ea59b0f6efe403aca2d0743e174da9bd2b7cf62f1e WatchSource:0}: Error finding container e6098c957732b30cbb41c5ea59b0f6efe403aca2d0743e174da9bd2b7cf62f1e: Status 404 returned error can't find the container with id e6098c957732b30cbb41c5ea59b0f6efe403aca2d0743e174da9bd2b7cf62f1e
Jul 23 16:18:39.425147 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:39.425110    2562 generic.go:356] "Generic (PLEG): container finished" podID="cd87dec5-f7e2-43a4-a2db-d5428f800f09" containerID="43a36673a99d162870b8cc39b80193abd4a1a097eca13ab28d21475c4dfc497c" exitCode=0
Jul 23 16:18:39.425608 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:39.425201    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx" event={"ID":"cd87dec5-f7e2-43a4-a2db-d5428f800f09","Type":"ContainerDied","Data":"43a36673a99d162870b8cc39b80193abd4a1a097eca13ab28d21475c4dfc497c"}
Jul 23 16:18:39.425608 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:39.425265    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx" event={"ID":"cd87dec5-f7e2-43a4-a2db-d5428f800f09","Type":"ContainerStarted","Data":"e6098c957732b30cbb41c5ea59b0f6efe403aca2d0743e174da9bd2b7cf62f1e"}
Jul 23 16:18:40.567273 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:40.567246    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"
Jul 23 16:18:40.654570 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:40.654527    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd87dec5-f7e2-43a4-a2db-d5428f800f09-maas-api-service-ca\") pod \"cd87dec5-f7e2-43a4-a2db-d5428f800f09\" (UID: \"cd87dec5-f7e2-43a4-a2db-d5428f800f09\") "
Jul 23 16:18:40.654961 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:40.654940    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd87dec5-f7e2-43a4-a2db-d5428f800f09-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cd87dec5-f7e2-43a4-a2db-d5428f800f09" (UID: "cd87dec5-f7e2-43a4-a2db-d5428f800f09"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:18:40.755754 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:40.755666    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd87dec5-f7e2-43a4-a2db-d5428f800f09-maas-api-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:18:41.437986 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:41.437954    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"
Jul 23 16:18:41.438157 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:41.437989    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx" event={"ID":"cd87dec5-f7e2-43a4-a2db-d5428f800f09","Type":"ContainerDied","Data":"e6098c957732b30cbb41c5ea59b0f6efe403aca2d0743e174da9bd2b7cf62f1e"}
Jul 23 16:18:41.438157 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:41.438066    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6098c957732b30cbb41c5ea59b0f6efe403aca2d0743e174da9bd2b7cf62f1e"
Jul 23 16:18:48.395254 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.395217    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"]
Jul 23 16:18:48.395759 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.395469    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc" podUID="87da44a5-ced4-466f-abb8-165c3f397932" containerName="maas-api" containerID="cri-o://e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0" gracePeriod=30
Jul 23 16:18:48.642845 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.642794    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:48.733035 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.732994    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87da44a5-ced4-466f-abb8-165c3f397932-openshift-service-ca\") pod \"87da44a5-ced4-466f-abb8-165c3f397932\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") "
Jul 23 16:18:48.733221 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.733048    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prf2m\" (UniqueName: \"kubernetes.io/projected/87da44a5-ced4-466f-abb8-165c3f397932-kube-api-access-prf2m\") pod \"87da44a5-ced4-466f-abb8-165c3f397932\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") "
Jul 23 16:18:48.733221 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.733085    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87da44a5-ced4-466f-abb8-165c3f397932-maas-api-tls\") pod \"87da44a5-ced4-466f-abb8-165c3f397932\" (UID: \"87da44a5-ced4-466f-abb8-165c3f397932\") "
Jul 23 16:18:48.733453 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.733427    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87da44a5-ced4-466f-abb8-165c3f397932-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87da44a5-ced4-466f-abb8-165c3f397932" (UID: "87da44a5-ced4-466f-abb8-165c3f397932"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:18:48.735222 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.735198    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87da44a5-ced4-466f-abb8-165c3f397932-kube-api-access-prf2m" (OuterVolumeSpecName: "kube-api-access-prf2m") pod "87da44a5-ced4-466f-abb8-165c3f397932" (UID: "87da44a5-ced4-466f-abb8-165c3f397932"). InnerVolumeSpecName "kube-api-access-prf2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:18:48.735277 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.735242    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87da44a5-ced4-466f-abb8-165c3f397932-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87da44a5-ced4-466f-abb8-165c3f397932" (UID: "87da44a5-ced4-466f-abb8-165c3f397932"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:18:48.834615 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.834573    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87da44a5-ced4-466f-abb8-165c3f397932-openshift-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:18:48.834615 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.834608    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prf2m\" (UniqueName: \"kubernetes.io/projected/87da44a5-ced4-466f-abb8-165c3f397932-kube-api-access-prf2m\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:18:48.834615 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:48.834617    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87da44a5-ced4-466f-abb8-165c3f397932-maas-api-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:18:49.483717 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:49.483678    2562 generic.go:356] "Generic (PLEG): container finished" podID="87da44a5-ced4-466f-abb8-165c3f397932" containerID="e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0" exitCode=0
Jul 23 16:18:49.484199 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:49.483723    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"
Jul 23 16:18:49.484199 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:49.483769    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc" event={"ID":"87da44a5-ced4-466f-abb8-165c3f397932","Type":"ContainerDied","Data":"e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0"}
Jul 23 16:18:49.484199 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:49.483850    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc" event={"ID":"87da44a5-ced4-466f-abb8-165c3f397932","Type":"ContainerDied","Data":"b3560a5c2ca9040a3d983e32b286e9109a0c077fdf53f9e4eba53e56107f9bb0"}
Jul 23 16:18:49.484199 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:49.483878    2562 scope.go:117] "RemoveContainer" containerID="e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0"
Jul 23 16:18:49.493323 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:49.493301    2562 scope.go:117] "RemoveContainer" containerID="e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0"
Jul 23 16:18:49.493604 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:18:49.493584    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0\": container with ID starting with e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0 not found: ID does not exist" containerID="e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0"
Jul 23 16:18:49.493680 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:49.493612    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0"} err="failed to get container status \"e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0\": rpc error: code = NotFound desc = could not find container \"e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0\": container with ID starting with e800e089eaa4f90b2c7100fa1f69fd46f2028163fe6bd58f4b6f417b27dc0ad0 not found: ID does not exist"
Jul 23 16:18:49.501415 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:49.501389    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"]
Jul 23 16:18:49.507150 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:49.507129    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f4ec811-6c6c7bbf88-gxnbc"]
Jul 23 16:18:51.254662 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:51.254623    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87da44a5-ced4-466f-abb8-165c3f397932" path="/var/lib/kubelet/pods/87da44a5-ced4-466f-abb8-165c3f397932/volumes"
Jul 23 16:18:53.065864 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:53.065830    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"]
Jul 23 16:18:53.070541 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:53.070516    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f4ec811-t9rqx"]
Jul 23 16:18:53.255125 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:53.255092    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd87dec5-f7e2-43a4-a2db-d5428f800f09" path="/var/lib/kubelet/pods/cd87dec5-f7e2-43a4-a2db-d5428f800f09/volumes"
Jul 23 16:18:57.016747 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.016708    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"]
Jul 23 16:18:57.017336 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.017318    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87da44a5-ced4-466f-abb8-165c3f397932" containerName="maas-api"
Jul 23 16:18:57.017336 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.017337    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="87da44a5-ced4-466f-abb8-165c3f397932" containerName="maas-api"
Jul 23 16:18:57.017443 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.017355    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd87dec5-f7e2-43a4-a2db-d5428f800f09" containerName="revoke-keys"
Jul 23 16:18:57.017443 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.017361    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd87dec5-f7e2-43a4-a2db-d5428f800f09" containerName="revoke-keys"
Jul 23 16:18:57.017443 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.017437    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="87da44a5-ced4-466f-abb8-165c3f397932" containerName="maas-api"
Jul 23 16:18:57.017555 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.017446    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd87dec5-f7e2-43a4-a2db-d5428f800f09" containerName="revoke-keys"
Jul 23 16:18:57.020469 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.020451    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:57.023294 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.023268    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e475fc66-serving-cert\""
Jul 23 16:18:57.023407 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.023354    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 16:18:57.024498 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.024480    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5j8mj\""
Jul 23 16:18:57.024588 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.024483    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 16:18:57.031774 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.031753    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"]
Jul 23 16:18:57.119722 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.119682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7bw6\" (UniqueName: \"kubernetes.io/projected/11c01ce9-2458-4bff-a2e5-ee113f74be6b-kube-api-access-c7bw6\") pod \"maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:57.119928 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.119745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11c01ce9-2458-4bff-a2e5-ee113f74be6b-maas-api-tls\") pod \"maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:57.119928 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.119773    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11c01ce9-2458-4bff-a2e5-ee113f74be6b-openshift-service-ca\") pod \"maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:57.220701 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.220660    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11c01ce9-2458-4bff-a2e5-ee113f74be6b-maas-api-tls\") pod \"maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:57.220701 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.220705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11c01ce9-2458-4bff-a2e5-ee113f74be6b-openshift-service-ca\") pod \"maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:57.220962 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.220779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7bw6\" (UniqueName: \"kubernetes.io/projected/11c01ce9-2458-4bff-a2e5-ee113f74be6b-kube-api-access-c7bw6\") pod \"maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:57.223988 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.223964    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 16:18:57.224146 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.224118    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e475fc66-serving-cert\""
Jul 23 16:18:57.230983 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:18:57.230958    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-serving-cert: secret "maas-api-e2e-ait-e475fc66-serving-cert" not found
Jul 23 16:18:57.231069 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:18:57.231034    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11c01ce9-2458-4bff-a2e5-ee113f74be6b-maas-api-tls podName:11c01ce9-2458-4bff-a2e5-ee113f74be6b nodeName:}" failed. No retries permitted until 2026-07-23 16:18:57.731017795 +0000 UTC m=+1800.935385167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11c01ce9-2458-4bff-a2e5-ee113f74be6b-maas-api-tls") pod "maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66" (UID: "11c01ce9-2458-4bff-a2e5-ee113f74be6b") : secret "maas-api-e2e-ait-e475fc66-serving-cert" not found
Jul 23 16:18:57.231218 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.231189    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 16:18:57.231556 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.231535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11c01ce9-2458-4bff-a2e5-ee113f74be6b-openshift-service-ca\") pod \"maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:57.242067 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.242039    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7bw6\" (UniqueName: \"kubernetes.io/projected/11c01ce9-2458-4bff-a2e5-ee113f74be6b-kube-api-access-c7bw6\") pod \"maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:57.277899 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.277784    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:18:57.279623 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.279596    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:18:57.826940 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.826900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11c01ce9-2458-4bff-a2e5-ee113f74be6b-maas-api-tls\") pod \"maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:57.829346 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.829314    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11c01ce9-2458-4bff-a2e5-ee113f74be6b-maas-api-tls\") pod \"maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:57.937731 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.937704    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5j8mj\""
Jul 23 16:18:57.943533 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:57.943508    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:58.089667 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:58.089641    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"]
Jul 23 16:18:58.091886 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:18:58.091851    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11c01ce9_2458_4bff_a2e5_ee113f74be6b.slice/crio-face8a0ba50eba131807f59f816c90f80ac83b272084c8e719d9ddf7269ee186 WatchSource:0}: Error finding container face8a0ba50eba131807f59f816c90f80ac83b272084c8e719d9ddf7269ee186: Status 404 returned error can't find the container with id face8a0ba50eba131807f59f816c90f80ac83b272084c8e719d9ddf7269ee186
Jul 23 16:18:58.535019 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:58.534984    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66" event={"ID":"11c01ce9-2458-4bff-a2e5-ee113f74be6b","Type":"ContainerStarted","Data":"face8a0ba50eba131807f59f816c90f80ac83b272084c8e719d9ddf7269ee186"}
Jul 23 16:18:59.541630 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:59.541592    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66" event={"ID":"11c01ce9-2458-4bff-a2e5-ee113f74be6b","Type":"ContainerStarted","Data":"5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92"}
Jul 23 16:18:59.542060 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:59.541655    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:18:59.560849 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:18:59.560765    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66" podStartSLOduration=2.8285201669999998 podStartE2EDuration="3.560744049s" totalImagesPullingTime="732.223882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:18:59.558171428 +0000 UTC m=+1802.762538821" watchObservedRunningTime="2026-07-23 16:18:59.560744049 +0000 UTC m=+1802.765111443"
Jul 23 16:19:05.555110 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:05.555074    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:19:22.915904 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:22.915867    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"]
Jul 23 16:19:22.919828 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:22.919792    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"
Jul 23 16:19:22.922692 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:22.922672    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w6jwk\""
Jul 23 16:19:22.927934 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:22.927907    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"]
Jul 23 16:19:22.969590 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:22.969555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4\" (UID: \"6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"
Jul 23 16:19:23.070853 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:23.070815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4\" (UID: \"6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"
Jul 23 16:19:23.071488 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:23.071466    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4\" (UID: \"6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"
Jul 23 16:19:23.233980 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:23.233944    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"
Jul 23 16:19:23.370543 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:23.370515    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"]
Jul 23 16:19:23.373388 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:19:23.373355    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6348d40e_3ef4_42a9_bbec_d69f2aaf5dc0.slice/crio-0a50329bb3755c5c9f044a17bcec0d6d25a2180eb1f2dcb6edf6e04006452d81 WatchSource:0}: Error finding container 0a50329bb3755c5c9f044a17bcec0d6d25a2180eb1f2dcb6edf6e04006452d81: Status 404 returned error can't find the container with id 0a50329bb3755c5c9f044a17bcec0d6d25a2180eb1f2dcb6edf6e04006452d81
Jul 23 16:19:23.678351 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:23.678305    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4" event={"ID":"6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0","Type":"ContainerStarted","Data":"bae557a2b9dddcc60bae8b26efe26289703010e64adae66c8f638a8811dda4e3"}
Jul 23 16:19:23.678525 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:23.678362    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4" event={"ID":"6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0","Type":"ContainerStarted","Data":"0a50329bb3755c5c9f044a17bcec0d6d25a2180eb1f2dcb6edf6e04006452d81"}
Jul 23 16:19:23.694981 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:23.694924    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4" podStartSLOduration=1.694908784 podStartE2EDuration="1.694908784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:19:23.692862234 +0000 UTC m=+1826.897229630" watchObservedRunningTime="2026-07-23 16:19:23.694908784 +0000 UTC m=+1826.899276177"
Jul 23 16:19:24.684938 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:24.684900    2562 generic.go:356] "Generic (PLEG): container finished" podID="6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0" containerID="bae557a2b9dddcc60bae8b26efe26289703010e64adae66c8f638a8811dda4e3" exitCode=0
Jul 23 16:19:24.685356 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:24.684993    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4" event={"ID":"6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0","Type":"ContainerDied","Data":"bae557a2b9dddcc60bae8b26efe26289703010e64adae66c8f638a8811dda4e3"}
Jul 23 16:19:25.820696 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:25.820672    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"
Jul 23 16:19:25.897154 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:25.897111    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0-maas-api-service-ca\") pod \"6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0\" (UID: \"6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0\") "
Jul 23 16:19:25.897528 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:25.897502    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0" (UID: "6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:19:25.998856 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:25.998754    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0-maas-api-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:19:26.702434 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:26.702407    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"
Jul 23 16:19:26.702624 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:26.702398    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4" event={"ID":"6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0","Type":"ContainerDied","Data":"0a50329bb3755c5c9f044a17bcec0d6d25a2180eb1f2dcb6edf6e04006452d81"}
Jul 23 16:19:26.702624 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:26.702557    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a50329bb3755c5c9f044a17bcec0d6d25a2180eb1f2dcb6edf6e04006452d81"
Jul 23 16:19:33.342024 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.341982    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"]
Jul 23 16:19:33.342594 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.342252    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66" podUID="11c01ce9-2458-4bff-a2e5-ee113f74be6b" containerName="maas-api" containerID="cri-o://5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92" gracePeriod=30
Jul 23 16:19:33.587590 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.587565    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:19:33.674661 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.674628    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7bw6\" (UniqueName: \"kubernetes.io/projected/11c01ce9-2458-4bff-a2e5-ee113f74be6b-kube-api-access-c7bw6\") pod \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") "
Jul 23 16:19:33.674904 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.674699    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11c01ce9-2458-4bff-a2e5-ee113f74be6b-maas-api-tls\") pod \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") "
Jul 23 16:19:33.674904 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.674731    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11c01ce9-2458-4bff-a2e5-ee113f74be6b-openshift-service-ca\") pod \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\" (UID: \"11c01ce9-2458-4bff-a2e5-ee113f74be6b\") "
Jul 23 16:19:33.675140 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.675114    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11c01ce9-2458-4bff-a2e5-ee113f74be6b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11c01ce9-2458-4bff-a2e5-ee113f74be6b" (UID: "11c01ce9-2458-4bff-a2e5-ee113f74be6b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:19:33.676914 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.676884    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11c01ce9-2458-4bff-a2e5-ee113f74be6b-kube-api-access-c7bw6" (OuterVolumeSpecName: "kube-api-access-c7bw6") pod "11c01ce9-2458-4bff-a2e5-ee113f74be6b" (UID: "11c01ce9-2458-4bff-a2e5-ee113f74be6b"). InnerVolumeSpecName "kube-api-access-c7bw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:19:33.676914 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.676886    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11c01ce9-2458-4bff-a2e5-ee113f74be6b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11c01ce9-2458-4bff-a2e5-ee113f74be6b" (UID: "11c01ce9-2458-4bff-a2e5-ee113f74be6b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:19:33.738828 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.738773    2562 generic.go:356] "Generic (PLEG): container finished" podID="11c01ce9-2458-4bff-a2e5-ee113f74be6b" containerID="5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92" exitCode=0
Jul 23 16:19:33.739028 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.738842    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"
Jul 23 16:19:33.739028 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.738872    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66" event={"ID":"11c01ce9-2458-4bff-a2e5-ee113f74be6b","Type":"ContainerDied","Data":"5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92"}
Jul 23 16:19:33.739028 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.738940    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66" event={"ID":"11c01ce9-2458-4bff-a2e5-ee113f74be6b","Type":"ContainerDied","Data":"face8a0ba50eba131807f59f816c90f80ac83b272084c8e719d9ddf7269ee186"}
Jul 23 16:19:33.739028 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.738973    2562 scope.go:117] "RemoveContainer" containerID="5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92"
Jul 23 16:19:33.748030 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.748015    2562 scope.go:117] "RemoveContainer" containerID="5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92"
Jul 23 16:19:33.748284 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:19:33.748265    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92\": container with ID starting with 5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92 not found: ID does not exist" containerID="5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92"
Jul 23 16:19:33.748330 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.748294    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92"} err="failed to get container status \"5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92\": rpc error: code = NotFound desc = could not find container \"5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92\": container with ID starting with 5d37aa0c518f178181163c2377f932c0760c7879751a0c0ea9aeefae0e1dae92 not found: ID does not exist"
Jul 23 16:19:33.761480 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.761443    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"]
Jul 23 16:19:33.768617 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.768591    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e475fc66-cdf7cc4c5-8nt66"]
Jul 23 16:19:33.776266 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.776246    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11c01ce9-2458-4bff-a2e5-ee113f74be6b-maas-api-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:19:33.776363 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.776269    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11c01ce9-2458-4bff-a2e5-ee113f74be6b-openshift-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:19:33.776363 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:33.776281    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7bw6\" (UniqueName: \"kubernetes.io/projected/11c01ce9-2458-4bff-a2e5-ee113f74be6b-kube-api-access-c7bw6\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:19:35.254158 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:35.254129    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11c01ce9-2458-4bff-a2e5-ee113f74be6b" path="/var/lib/kubelet/pods/11c01ce9-2458-4bff-a2e5-ee113f74be6b/volumes"
Jul 23 16:19:38.016576 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:38.016533    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"]
Jul 23 16:19:38.020607 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:38.020582    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e475fc66-mrfs4"]
Jul 23 16:19:39.254761 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:39.254724    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0" path="/var/lib/kubelet/pods/6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0/volumes"
Jul 23 16:19:42.320723 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.320684    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"]
Jul 23 16:19:42.321159 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.321145    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11c01ce9-2458-4bff-a2e5-ee113f74be6b" containerName="maas-api"
Jul 23 16:19:42.321214 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.321160    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="11c01ce9-2458-4bff-a2e5-ee113f74be6b" containerName="maas-api"
Jul 23 16:19:42.321214 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.321170    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0" containerName="revoke-keys"
Jul 23 16:19:42.321214 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.321176    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0" containerName="revoke-keys"
Jul 23 16:19:42.321370 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.321251    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6348d40e-3ef4-42a9-bbec-d69f2aaf5dc0" containerName="revoke-keys"
Jul 23 16:19:42.321370 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.321264    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="11c01ce9-2458-4bff-a2e5-ee113f74be6b" containerName="maas-api"
Jul 23 16:19:42.325896 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.325873    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:42.329619 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.329593    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-063a9ab3-serving-cert\""
Jul 23 16:19:42.329759 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.329599    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 16:19:42.329759 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.329694    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5j8mj\""
Jul 23 16:19:42.331019 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.330997    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 16:19:42.339910 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.339884    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"]
Jul 23 16:19:42.463037 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.462997    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls\") pod \"maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:42.463220 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.463156    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h949b\" (UniqueName: \"kubernetes.io/projected/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-kube-api-access-h949b\") pod \"maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:42.463220 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.463189    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-openshift-service-ca\") pod \"maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:42.563886 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.563850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h949b\" (UniqueName: \"kubernetes.io/projected/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-kube-api-access-h949b\") pod \"maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:42.564058 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.563898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-openshift-service-ca\") pod \"maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:42.564058 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.563955    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls\") pod \"maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:42.564226 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:19:42.564164    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-serving-cert: secret "maas-api-e2e-derive-063a9ab3-serving-cert" not found
Jul 23 16:19:42.564282 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:19:42.564257    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls podName:16dd7f81-95e1-47f2-ba6b-39f7c72845d0 nodeName:}" failed. No retries permitted until 2026-07-23 16:19:43.064233743 +0000 UTC m=+1846.268601118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls") pod "maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4" (UID: "16dd7f81-95e1-47f2-ba6b-39f7c72845d0") : secret "maas-api-e2e-derive-063a9ab3-serving-cert" not found
Jul 23 16:19:42.564566 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.564544    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-openshift-service-ca\") pod \"maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:42.576639 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:42.576575    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h949b\" (UniqueName: \"kubernetes.io/projected/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-kube-api-access-h949b\") pod \"maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:43.069546 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:43.069508    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls\") pod \"maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:43.069734 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:19:43.069668    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-serving-cert: secret "maas-api-e2e-derive-063a9ab3-serving-cert" not found
Jul 23 16:19:43.069734 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:19:43.069731    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls podName:16dd7f81-95e1-47f2-ba6b-39f7c72845d0 nodeName:}" failed. No retries permitted until 2026-07-23 16:19:44.069714302 +0000 UTC m=+1847.274081673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls") pod "maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4" (UID: "16dd7f81-95e1-47f2-ba6b-39f7c72845d0") : secret "maas-api-e2e-derive-063a9ab3-serving-cert" not found
Jul 23 16:19:44.080265 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:44.080228    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls\") pod \"maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:44.082631 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:44.082609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls\") pod \"maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:44.137767 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:44.137722    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:44.278393 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:44.278367    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"]
Jul 23 16:19:44.279931 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:19:44.279902    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16dd7f81_95e1_47f2_ba6b_39f7c72845d0.slice/crio-6cf5d997c90812a098972758137be210982abdbde99c46024fbda1cc14a3c05e WatchSource:0}: Error finding container 6cf5d997c90812a098972758137be210982abdbde99c46024fbda1cc14a3c05e: Status 404 returned error can't find the container with id 6cf5d997c90812a098972758137be210982abdbde99c46024fbda1cc14a3c05e
Jul 23 16:19:44.803377 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:44.803340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4" event={"ID":"16dd7f81-95e1-47f2-ba6b-39f7c72845d0","Type":"ContainerStarted","Data":"6cf5d997c90812a098972758137be210982abdbde99c46024fbda1cc14a3c05e"}
Jul 23 16:19:45.809998 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:45.809962    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4" event={"ID":"16dd7f81-95e1-47f2-ba6b-39f7c72845d0","Type":"ContainerStarted","Data":"93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe"}
Jul 23 16:19:45.810390 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:45.810026    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:19:45.827237 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:45.827165    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4" podStartSLOduration=3.183894035 podStartE2EDuration="3.827117741s" totalImagesPullingTime="643.223706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:19:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:19:45.825901606 +0000 UTC m=+1849.030269065" watchObservedRunningTime="2026-07-23 16:19:45.827117741 +0000 UTC m=+1849.031485134"
Jul 23 16:19:51.822982 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:19:51.822955    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:20:07.098993 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.098949    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"]
Jul 23 16:20:07.102850 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.102829    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"
Jul 23 16:20:07.105490 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.105467    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w6jwk\""
Jul 23 16:20:07.110565 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.110399    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"]
Jul 23 16:20:07.208915 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.208880    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dea4b82-ba01-4b6a-a4d4-05dfc5af65df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv\" (UID: \"0dea4b82-ba01-4b6a-a4d4-05dfc5af65df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"
Jul 23 16:20:07.310402 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.310364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dea4b82-ba01-4b6a-a4d4-05dfc5af65df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv\" (UID: \"0dea4b82-ba01-4b6a-a4d4-05dfc5af65df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"
Jul 23 16:20:07.311218 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.311191    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dea4b82-ba01-4b6a-a4d4-05dfc5af65df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv\" (UID: \"0dea4b82-ba01-4b6a-a4d4-05dfc5af65df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"
Jul 23 16:20:07.414852 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.414745    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"
Jul 23 16:20:07.602765 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.602663    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"]
Jul 23 16:20:07.605161 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:20:07.605123    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dea4b82_ba01_4b6a_a4d4_05dfc5af65df.slice/crio-328358cd93ed3a2a00492d89900c223ab59b76d07e8160d32bcbf3f09e0a7c39 WatchSource:0}: Error finding container 328358cd93ed3a2a00492d89900c223ab59b76d07e8160d32bcbf3f09e0a7c39: Status 404 returned error can't find the container with id 328358cd93ed3a2a00492d89900c223ab59b76d07e8160d32bcbf3f09e0a7c39
Jul 23 16:20:07.937710 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.937672    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv" event={"ID":"0dea4b82-ba01-4b6a-a4d4-05dfc5af65df","Type":"ContainerStarted","Data":"a07a624fc99e29de7f1eaa0addbf8f8adc64c06262559a28b07ec661e90f64bd"}
Jul 23 16:20:07.937919 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.937732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv" event={"ID":"0dea4b82-ba01-4b6a-a4d4-05dfc5af65df","Type":"ContainerStarted","Data":"328358cd93ed3a2a00492d89900c223ab59b76d07e8160d32bcbf3f09e0a7c39"}
Jul 23 16:20:07.957015 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:07.956959    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv" podStartSLOduration=0.956944345 podStartE2EDuration="956.944345ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:20:07.955624187 +0000 UTC m=+1871.159991581" watchObservedRunningTime="2026-07-23 16:20:07.956944345 +0000 UTC m=+1871.161311738"
Jul 23 16:20:08.944315 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:08.944284    2562 generic.go:356] "Generic (PLEG): container finished" podID="0dea4b82-ba01-4b6a-a4d4-05dfc5af65df" containerID="a07a624fc99e29de7f1eaa0addbf8f8adc64c06262559a28b07ec661e90f64bd" exitCode=0
Jul 23 16:20:08.944742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:08.944391    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv" event={"ID":"0dea4b82-ba01-4b6a-a4d4-05dfc5af65df","Type":"ContainerDied","Data":"a07a624fc99e29de7f1eaa0addbf8f8adc64c06262559a28b07ec661e90f64bd"}
Jul 23 16:20:10.086972 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:10.086946    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"
Jul 23 16:20:10.236695 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:10.236608    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dea4b82-ba01-4b6a-a4d4-05dfc5af65df-maas-api-service-ca\") pod \"0dea4b82-ba01-4b6a-a4d4-05dfc5af65df\" (UID: \"0dea4b82-ba01-4b6a-a4d4-05dfc5af65df\") "
Jul 23 16:20:10.237009 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:10.236986    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dea4b82-ba01-4b6a-a4d4-05dfc5af65df-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0dea4b82-ba01-4b6a-a4d4-05dfc5af65df" (UID: "0dea4b82-ba01-4b6a-a4d4-05dfc5af65df"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:20:10.338302 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:10.338248    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dea4b82-ba01-4b6a-a4d4-05dfc5af65df-maas-api-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:20:10.955880 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:10.955852    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"
Jul 23 16:20:10.955880 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:10.955861    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv" event={"ID":"0dea4b82-ba01-4b6a-a4d4-05dfc5af65df","Type":"ContainerDied","Data":"328358cd93ed3a2a00492d89900c223ab59b76d07e8160d32bcbf3f09e0a7c39"}
Jul 23 16:20:10.956103 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:10.955903    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="328358cd93ed3a2a00492d89900c223ab59b76d07e8160d32bcbf3f09e0a7c39"
Jul 23 16:20:17.519651 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.519495    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"]
Jul 23 16:20:17.520171 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.519737    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4" podUID="16dd7f81-95e1-47f2-ba6b-39f7c72845d0" containerName="maas-api" containerID="cri-o://93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe" gracePeriod=30
Jul 23 16:20:17.771899 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.771841    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:20:17.912632 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.912595    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h949b\" (UniqueName: \"kubernetes.io/projected/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-kube-api-access-h949b\") pod \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") "
Jul 23 16:20:17.912844 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.912649    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls\") pod \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") "
Jul 23 16:20:17.912844 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.912773    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-openshift-service-ca\") pod \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\" (UID: \"16dd7f81-95e1-47f2-ba6b-39f7c72845d0\") "
Jul 23 16:20:17.913142 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.913111    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16dd7f81-95e1-47f2-ba6b-39f7c72845d0" (UID: "16dd7f81-95e1-47f2-ba6b-39f7c72845d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:20:17.914999 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.914976    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-kube-api-access-h949b" (OuterVolumeSpecName: "kube-api-access-h949b") pod "16dd7f81-95e1-47f2-ba6b-39f7c72845d0" (UID: "16dd7f81-95e1-47f2-ba6b-39f7c72845d0"). InnerVolumeSpecName "kube-api-access-h949b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:20:17.915122 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.915097    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16dd7f81-95e1-47f2-ba6b-39f7c72845d0" (UID: "16dd7f81-95e1-47f2-ba6b-39f7c72845d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:20:17.997055 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.997025    2562 generic.go:356] "Generic (PLEG): container finished" podID="16dd7f81-95e1-47f2-ba6b-39f7c72845d0" containerID="93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe" exitCode=0
Jul 23 16:20:17.997263 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.997069    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"
Jul 23 16:20:17.997263 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.997119    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4" event={"ID":"16dd7f81-95e1-47f2-ba6b-39f7c72845d0","Type":"ContainerDied","Data":"93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe"}
Jul 23 16:20:17.997263 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.997182    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4" event={"ID":"16dd7f81-95e1-47f2-ba6b-39f7c72845d0","Type":"ContainerDied","Data":"6cf5d997c90812a098972758137be210982abdbde99c46024fbda1cc14a3c05e"}
Jul 23 16:20:17.997263 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:17.997218    2562 scope.go:117] "RemoveContainer" containerID="93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe"
Jul 23 16:20:18.006944 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:18.006924    2562 scope.go:117] "RemoveContainer" containerID="93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe"
Jul 23 16:20:18.007216 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:20:18.007193    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe\": container with ID starting with 93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe not found: ID does not exist" containerID="93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe"
Jul 23 16:20:18.007276 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:18.007226    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe"} err="failed to get container status \"93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe\": rpc error: code = NotFound desc = could not find container \"93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe\": container with ID starting with 93ad928dac9394636de4fd4468bbfe01d4181ae32def6181a645478f0cf447fe not found: ID does not exist"
Jul 23 16:20:18.013815 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:18.013781    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-openshift-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:20:18.013908 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:18.013817    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h949b\" (UniqueName: \"kubernetes.io/projected/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-kube-api-access-h949b\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:20:18.013908 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:18.013828    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd7f81-95e1-47f2-ba6b-39f7c72845d0-maas-api-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:20:18.019889 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:18.019866    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"]
Jul 23 16:20:18.024748 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:18.024694    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-063a9ab3-6968488d97-6zpb4"]
Jul 23 16:20:19.254864 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:19.254772    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16dd7f81-95e1-47f2-ba6b-39f7c72845d0" path="/var/lib/kubelet/pods/16dd7f81-95e1-47f2-ba6b-39f7c72845d0/volumes"
Jul 23 16:20:22.180390 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:22.180352    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"]
Jul 23 16:20:22.182825 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:22.182788    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-063a9ab3-sbcjv"]
Jul 23 16:20:23.253993 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:20:23.253963    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dea4b82-ba01-4b6a-a4d4-05dfc5af65df" path="/var/lib/kubelet/pods/0dea4b82-ba01-4b6a-a4d4-05dfc5af65df/volumes"
Jul 23 16:22:09.129062 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.129023    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"]
Jul 23 16:22:09.129571 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.129474    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16dd7f81-95e1-47f2-ba6b-39f7c72845d0" containerName="maas-api"
Jul 23 16:22:09.129571 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.129488    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="16dd7f81-95e1-47f2-ba6b-39f7c72845d0" containerName="maas-api"
Jul 23 16:22:09.129571 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.129497    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dea4b82-ba01-4b6a-a4d4-05dfc5af65df" containerName="revoke-keys"
Jul 23 16:22:09.129571 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.129503    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dea4b82-ba01-4b6a-a4d4-05dfc5af65df" containerName="revoke-keys"
Jul 23 16:22:09.129771 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.129575    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dea4b82-ba01-4b6a-a4d4-05dfc5af65df" containerName="revoke-keys"
Jul 23 16:22:09.129771 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.129586    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="16dd7f81-95e1-47f2-ba6b-39f7c72845d0" containerName="maas-api"
Jul 23 16:22:09.132235 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.132215    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:09.135790 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.135645    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 16:22:09.135790 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.135660    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5j8mj\""
Jul 23 16:22:09.135790 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.135687    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 16:22:09.135790 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.135645    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2b5c55cc-serving-cert\""
Jul 23 16:22:09.136517 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.136481    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"]
Jul 23 16:22:09.296510 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.296465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qn56\" (UniqueName: \"kubernetes.io/projected/2f46082f-d298-4ea2-978a-42de7ddf32f5-kube-api-access-5qn56\") pod \"maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:09.296704 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.296522    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f46082f-d298-4ea2-978a-42de7ddf32f5-openshift-service-ca\") pod \"maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:09.296704 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.296634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f46082f-d298-4ea2-978a-42de7ddf32f5-maas-api-tls\") pod \"maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:09.397848 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.397719    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f46082f-d298-4ea2-978a-42de7ddf32f5-maas-api-tls\") pod \"maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:09.398040 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.397944    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qn56\" (UniqueName: \"kubernetes.io/projected/2f46082f-d298-4ea2-978a-42de7ddf32f5-kube-api-access-5qn56\") pod \"maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:09.398040 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.398017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f46082f-d298-4ea2-978a-42de7ddf32f5-openshift-service-ca\") pod \"maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:09.398673 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.398645    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f46082f-d298-4ea2-978a-42de7ddf32f5-openshift-service-ca\") pod \"maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:09.400308 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.400275    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f46082f-d298-4ea2-978a-42de7ddf32f5-maas-api-tls\") pod \"maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:09.407063 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.407041    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qn56\" (UniqueName: \"kubernetes.io/projected/2f46082f-d298-4ea2-978a-42de7ddf32f5-kube-api-access-5qn56\") pod \"maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:09.445023 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.444986    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:09.593552 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:09.593528    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"]
Jul 23 16:22:09.595465 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:22:09.595434    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f46082f_d298_4ea2_978a_42de7ddf32f5.slice/crio-e487668b68fc83af87e839c591ed988623dac023d58eecc845814c463f0d0845 WatchSource:0}: Error finding container e487668b68fc83af87e839c591ed988623dac023d58eecc845814c463f0d0845: Status 404 returned error can't find the container with id e487668b68fc83af87e839c591ed988623dac023d58eecc845814c463f0d0845
Jul 23 16:22:10.588661 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:10.588613    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt" event={"ID":"2f46082f-d298-4ea2-978a-42de7ddf32f5","Type":"ContainerStarted","Data":"e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63"}
Jul 23 16:22:10.589078 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:10.588691    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:10.589078 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:10.588716    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt" event={"ID":"2f46082f-d298-4ea2-978a-42de7ddf32f5","Type":"ContainerStarted","Data":"e487668b68fc83af87e839c591ed988623dac023d58eecc845814c463f0d0845"}
Jul 23 16:22:10.638005 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:10.637950    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt" podStartSLOduration=1.090586105 podStartE2EDuration="1.637921181s" totalImagesPullingTime="547.335076ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:22:10.636240735 +0000 UTC m=+1993.840608128" watchObservedRunningTime="2026-07-23 16:22:10.637921181 +0000 UTC m=+1993.842288574"
Jul 23 16:22:16.598929 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:16.598896    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:20.665218 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:20.665167    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"]
Jul 23 16:22:20.669315 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:20.669290    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"
Jul 23 16:22:20.673715 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:20.673684    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w6jwk\""
Jul 23 16:22:20.677885 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:20.677695    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"]
Jul 23 16:22:20.710354 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:20.710315    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1579f398-7d1f-483c-ab84-564bcefa9530-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62\" (UID: \"1579f398-7d1f-483c-ab84-564bcefa9530\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"
Jul 23 16:22:20.811710 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:20.811675    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1579f398-7d1f-483c-ab84-564bcefa9530-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62\" (UID: \"1579f398-7d1f-483c-ab84-564bcefa9530\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"
Jul 23 16:22:20.812329 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:20.812311    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1579f398-7d1f-483c-ab84-564bcefa9530-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62\" (UID: \"1579f398-7d1f-483c-ab84-564bcefa9530\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"
Jul 23 16:22:20.985315 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:20.985275    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"
Jul 23 16:22:21.129293 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:21.129248    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"]
Jul 23 16:22:21.132050 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:22:21.132019    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1579f398_7d1f_483c_ab84_564bcefa9530.slice/crio-ca0d6615c9a8528ed0fcb104c93fdbb78ce5d907511e5580d16a0135f7471fb7 WatchSource:0}: Error finding container ca0d6615c9a8528ed0fcb104c93fdbb78ce5d907511e5580d16a0135f7471fb7: Status 404 returned error can't find the container with id ca0d6615c9a8528ed0fcb104c93fdbb78ce5d907511e5580d16a0135f7471fb7
Jul 23 16:22:21.654551 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:21.654509    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62" event={"ID":"1579f398-7d1f-483c-ab84-564bcefa9530","Type":"ContainerStarted","Data":"bf019921f7335e28d328cb819916f14201288d1def5e1beb534190704ba650d1"}
Jul 23 16:22:21.654751 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:21.654584    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62" event={"ID":"1579f398-7d1f-483c-ab84-564bcefa9530","Type":"ContainerStarted","Data":"ca0d6615c9a8528ed0fcb104c93fdbb78ce5d907511e5580d16a0135f7471fb7"}
Jul 23 16:22:21.672152 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:21.672097    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62" podStartSLOduration=1.672079715 podStartE2EDuration="1.672079715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:22:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:22:21.668694934 +0000 UTC m=+2004.873062328" watchObservedRunningTime="2026-07-23 16:22:21.672079715 +0000 UTC m=+2004.876447108"
Jul 23 16:22:22.661740 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:22.661703    2562 generic.go:356] "Generic (PLEG): container finished" podID="1579f398-7d1f-483c-ab84-564bcefa9530" containerID="bf019921f7335e28d328cb819916f14201288d1def5e1beb534190704ba650d1" exitCode=0
Jul 23 16:22:22.661973 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:22.661787    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62" event={"ID":"1579f398-7d1f-483c-ab84-564bcefa9530","Type":"ContainerDied","Data":"bf019921f7335e28d328cb819916f14201288d1def5e1beb534190704ba650d1"}
Jul 23 16:22:23.807104 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:23.807076    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"
Jul 23 16:22:23.838655 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:23.838626    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1579f398-7d1f-483c-ab84-564bcefa9530-maas-api-service-ca\") pod \"1579f398-7d1f-483c-ab84-564bcefa9530\" (UID: \"1579f398-7d1f-483c-ab84-564bcefa9530\") "
Jul 23 16:22:23.838980 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:23.838960    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1579f398-7d1f-483c-ab84-564bcefa9530-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1579f398-7d1f-483c-ab84-564bcefa9530" (UID: "1579f398-7d1f-483c-ab84-564bcefa9530"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:22:23.939380 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:23.939344    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1579f398-7d1f-483c-ab84-564bcefa9530-maas-api-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:22:24.673751 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:24.673718    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"
Jul 23 16:22:24.673957 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:24.673753    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62" event={"ID":"1579f398-7d1f-483c-ab84-564bcefa9530","Type":"ContainerDied","Data":"ca0d6615c9a8528ed0fcb104c93fdbb78ce5d907511e5580d16a0135f7471fb7"}
Jul 23 16:22:24.673957 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:24.673828    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca0d6615c9a8528ed0fcb104c93fdbb78ce5d907511e5580d16a0135f7471fb7"
Jul 23 16:22:31.222035 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.222004    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"]
Jul 23 16:22:31.222861 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.222782    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt" podUID="2f46082f-d298-4ea2-978a-42de7ddf32f5" containerName="maas-api" containerID="cri-o://e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63" gracePeriod=30
Jul 23 16:22:31.480933 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.480862    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:31.509544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.509513    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f46082f-d298-4ea2-978a-42de7ddf32f5-maas-api-tls\") pod \"2f46082f-d298-4ea2-978a-42de7ddf32f5\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") "
Jul 23 16:22:31.509732 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.509576    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f46082f-d298-4ea2-978a-42de7ddf32f5-openshift-service-ca\") pod \"2f46082f-d298-4ea2-978a-42de7ddf32f5\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") "
Jul 23 16:22:31.509732 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.509638    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qn56\" (UniqueName: \"kubernetes.io/projected/2f46082f-d298-4ea2-978a-42de7ddf32f5-kube-api-access-5qn56\") pod \"2f46082f-d298-4ea2-978a-42de7ddf32f5\" (UID: \"2f46082f-d298-4ea2-978a-42de7ddf32f5\") "
Jul 23 16:22:31.509945 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.509918    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f46082f-d298-4ea2-978a-42de7ddf32f5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f46082f-d298-4ea2-978a-42de7ddf32f5" (UID: "2f46082f-d298-4ea2-978a-42de7ddf32f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:22:31.510048 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.510032    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f46082f-d298-4ea2-978a-42de7ddf32f5-openshift-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:22:31.511665 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.511645    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f46082f-d298-4ea2-978a-42de7ddf32f5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f46082f-d298-4ea2-978a-42de7ddf32f5" (UID: "2f46082f-d298-4ea2-978a-42de7ddf32f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:22:31.511936 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.511786    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f46082f-d298-4ea2-978a-42de7ddf32f5-kube-api-access-5qn56" (OuterVolumeSpecName: "kube-api-access-5qn56") pod "2f46082f-d298-4ea2-978a-42de7ddf32f5" (UID: "2f46082f-d298-4ea2-978a-42de7ddf32f5"). InnerVolumeSpecName "kube-api-access-5qn56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:22:31.610753 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.610714    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qn56\" (UniqueName: \"kubernetes.io/projected/2f46082f-d298-4ea2-978a-42de7ddf32f5-kube-api-access-5qn56\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:22:31.610753 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.610745    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f46082f-d298-4ea2-978a-42de7ddf32f5-maas-api-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:22:31.713415 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.713380    2562 generic.go:356] "Generic (PLEG): container finished" podID="2f46082f-d298-4ea2-978a-42de7ddf32f5" containerID="e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63" exitCode=0
Jul 23 16:22:31.713590 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.713429    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"
Jul 23 16:22:31.713590 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.713465    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt" event={"ID":"2f46082f-d298-4ea2-978a-42de7ddf32f5","Type":"ContainerDied","Data":"e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63"}
Jul 23 16:22:31.713590 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.713519    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt" event={"ID":"2f46082f-d298-4ea2-978a-42de7ddf32f5","Type":"ContainerDied","Data":"e487668b68fc83af87e839c591ed988623dac023d58eecc845814c463f0d0845"}
Jul 23 16:22:31.713590 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.713543    2562 scope.go:117] "RemoveContainer" containerID="e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63"
Jul 23 16:22:31.723509 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.723489    2562 scope.go:117] "RemoveContainer" containerID="e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63"
Jul 23 16:22:31.723783 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:22:31.723759    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63\": container with ID starting with e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63 not found: ID does not exist" containerID="e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63"
Jul 23 16:22:31.723932 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.723792    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63"} err="failed to get container status \"e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63\": rpc error: code = NotFound desc = could not find container \"e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63\": container with ID starting with e6869c308c1898469b024ade48056fdb3cc9a6c1b6d8a9d99d0c3dc3d5243c63 not found: ID does not exist"
Jul 23 16:22:31.741019 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.740940    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"]
Jul 23 16:22:31.745004 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:31.744974    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b5c55cc-7768dfbbd8-bwxlt"]
Jul 23 16:22:33.254464 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:33.254429    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f46082f-d298-4ea2-978a-42de7ddf32f5" path="/var/lib/kubelet/pods/2f46082f-d298-4ea2-978a-42de7ddf32f5/volumes"
Jul 23 16:22:35.751404 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:35.751365    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"]
Jul 23 16:22:35.753879 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:35.753852    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b5c55cc-xqr62"]
Jul 23 16:22:37.254630 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:22:37.254588    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1579f398-7d1f-483c-ab84-564bcefa9530" path="/var/lib/kubelet/pods/1579f398-7d1f-483c-ab84-564bcefa9530/volumes"
Jul 23 16:23:05.786387 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.786354    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"]
Jul 23 16:23:05.786882 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.786773    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1579f398-7d1f-483c-ab84-564bcefa9530" containerName="revoke-keys"
Jul 23 16:23:05.786882 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.786785    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1579f398-7d1f-483c-ab84-564bcefa9530" containerName="revoke-keys"
Jul 23 16:23:05.786882 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.786793    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f46082f-d298-4ea2-978a-42de7ddf32f5" containerName="maas-api"
Jul 23 16:23:05.786882 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.786829    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f46082f-d298-4ea2-978a-42de7ddf32f5" containerName="maas-api"
Jul 23 16:23:05.787059 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.786903    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f46082f-d298-4ea2-978a-42de7ddf32f5" containerName="maas-api"
Jul 23 16:23:05.787059 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.786915    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1579f398-7d1f-483c-ab84-564bcefa9530" containerName="revoke-keys"
Jul 23 16:23:05.790060 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.790039    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:05.792792 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.792769    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 16:23:05.792949 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.792879    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5j8mj\""
Jul 23 16:23:05.794040 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.794025    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 16:23:05.797160 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.796869    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4b72f23f-serving-cert\""
Jul 23 16:23:05.805569 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.805547    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"]
Jul 23 16:23:05.846928 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.846893    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cf8dbab-d242-4680-b9b6-95f33dac9896-openshift-service-ca\") pod \"maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:05.847093 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.846944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cf8dbab-d242-4680-b9b6-95f33dac9896-maas-api-tls\") pod \"maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:05.847093 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.847067    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8q6p\" (UniqueName: \"kubernetes.io/projected/6cf8dbab-d242-4680-b9b6-95f33dac9896-kube-api-access-z8q6p\") pod \"maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:05.947973 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.947932    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cf8dbab-d242-4680-b9b6-95f33dac9896-openshift-service-ca\") pod \"maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:05.948163 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.947981    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cf8dbab-d242-4680-b9b6-95f33dac9896-maas-api-tls\") pod \"maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:05.948163 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.948063    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8q6p\" (UniqueName: \"kubernetes.io/projected/6cf8dbab-d242-4680-b9b6-95f33dac9896-kube-api-access-z8q6p\") pod \"maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:05.948163 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:23:05.948144    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-serving-cert: secret "maas-api-e2e-mt-4b72f23f-serving-cert" not found
Jul 23 16:23:05.948283 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:23:05.948222    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cf8dbab-d242-4680-b9b6-95f33dac9896-maas-api-tls podName:6cf8dbab-d242-4680-b9b6-95f33dac9896 nodeName:}" failed. No retries permitted until 2026-07-23 16:23:06.448202634 +0000 UTC m=+2049.652570010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6cf8dbab-d242-4680-b9b6-95f33dac9896-maas-api-tls") pod "maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm" (UID: "6cf8dbab-d242-4680-b9b6-95f33dac9896") : secret "maas-api-e2e-mt-4b72f23f-serving-cert" not found
Jul 23 16:23:05.948629 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.948608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cf8dbab-d242-4680-b9b6-95f33dac9896-openshift-service-ca\") pod \"maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:05.974260 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:05.974228    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8q6p\" (UniqueName: \"kubernetes.io/projected/6cf8dbab-d242-4680-b9b6-95f33dac9896-kube-api-access-z8q6p\") pod \"maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:06.454272 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:06.454227    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cf8dbab-d242-4680-b9b6-95f33dac9896-maas-api-tls\") pod \"maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:06.457146 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:06.457108    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cf8dbab-d242-4680-b9b6-95f33dac9896-maas-api-tls\") pod \"maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:06.700579 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:06.700549    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:06.842534 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:06.842506    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"]
Jul 23 16:23:06.844753 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:23:06.844721    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cf8dbab_d242_4680_b9b6_95f33dac9896.slice/crio-3b263572d2a7aea55d769069e3f3bbfb956ad903a79a26c5063f785d8e875822 WatchSource:0}: Error finding container 3b263572d2a7aea55d769069e3f3bbfb956ad903a79a26c5063f785d8e875822: Status 404 returned error can't find the container with id 3b263572d2a7aea55d769069e3f3bbfb956ad903a79a26c5063f785d8e875822
Jul 23 16:23:06.846175 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:06.846156    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 16:23:06.924258 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:06.924217    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm" event={"ID":"6cf8dbab-d242-4680-b9b6-95f33dac9896","Type":"ContainerStarted","Data":"3b263572d2a7aea55d769069e3f3bbfb956ad903a79a26c5063f785d8e875822"}
Jul 23 16:23:07.931560 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:07.931525    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm" event={"ID":"6cf8dbab-d242-4680-b9b6-95f33dac9896","Type":"ContainerStarted","Data":"b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887"}
Jul 23 16:23:07.931990 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:07.931660    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:07.968238 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:07.968180    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm" podStartSLOduration=2.242536812 podStartE2EDuration="2.968161267s" totalImagesPullingTime="725.624455ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:23:07.966956968 +0000 UTC m=+2051.171324392" watchObservedRunningTime="2026-07-23 16:23:07.968161267 +0000 UTC m=+2051.172528661"
Jul 23 16:23:13.942419 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:13.942386    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:23:22.391443 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.391365    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"]
Jul 23 16:23:22.395961 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.395936    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.398792 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.398768    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4b72f23f\"/\"openshift-service-ca.crt\""
Jul 23 16:23:22.400255 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.400227    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4b72f23f\"/\"e2e-lifecycle-model-4b72f23f-kserve-self-signed-certs\""
Jul 23 16:23:22.400401 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.400270    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4b72f23f\"/\"kube-root-ca.crt\""
Jul 23 16:23:22.400453 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.400432    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4b72f23f\"/\"default-dockercfg-cr6r2\""
Jul 23 16:23:22.415314 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.415282    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"]
Jul 23 16:23:22.510379 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.510337    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-home\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.510379 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.510381    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-dshm\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.510679 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.510414    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrmfr\" (UniqueName: \"kubernetes.io/projected/80685dc9-6134-4a70-aa03-fc767be729fc-kube-api-access-zrmfr\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.510679 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.510537    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-tmp-dir\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.510679 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.510583    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-model-cache\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.510679 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.510639    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80685dc9-6134-4a70-aa03-fc767be729fc-tls-certs\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.611362 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.611325    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrmfr\" (UniqueName: \"kubernetes.io/projected/80685dc9-6134-4a70-aa03-fc767be729fc-kube-api-access-zrmfr\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.611569 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.611378    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-tmp-dir\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.611569 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.611401    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-model-cache\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.611569 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.611430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80685dc9-6134-4a70-aa03-fc767be729fc-tls-certs\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.611569 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.611470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-home\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.611569 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.611488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-dshm\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.611889 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.611750    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-tmp-dir\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.611889 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.611782    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-model-cache\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.611889 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.611879    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-home\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.613856 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.613830    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-dshm\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.614046 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.614029    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80685dc9-6134-4a70-aa03-fc767be729fc-tls-certs\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.619755 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.619732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrmfr\" (UniqueName: \"kubernetes.io/projected/80685dc9-6134-4a70-aa03-fc767be729fc-kube-api-access-zrmfr\") pod \"e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") " pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.708605 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.708560    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:22.850087 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:22.850056    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"]
Jul 23 16:23:22.851772 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:23:22.851739    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80685dc9_6134_4a70_aa03_fc767be729fc.slice/crio-1fd3a5be96c9b95185fbab97b998cc554f2f662a3930fcf07de70b53b18b4808 WatchSource:0}: Error finding container 1fd3a5be96c9b95185fbab97b998cc554f2f662a3930fcf07de70b53b18b4808: Status 404 returned error can't find the container with id 1fd3a5be96c9b95185fbab97b998cc554f2f662a3930fcf07de70b53b18b4808
Jul 23 16:23:23.020322 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:23.020227    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5" event={"ID":"80685dc9-6134-4a70-aa03-fc767be729fc","Type":"ContainerStarted","Data":"cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d"}
Jul 23 16:23:23.020322 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:23.020277    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5" event={"ID":"80685dc9-6134-4a70-aa03-fc767be729fc","Type":"ContainerStarted","Data":"1fd3a5be96c9b95185fbab97b998cc554f2f662a3930fcf07de70b53b18b4808"}
Jul 23 16:23:23.042975 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:23.042918    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5" podStartSLOduration=1.04290018 podStartE2EDuration="1.04290018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:23:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:23:23.041512052 +0000 UTC m=+2066.245879447" watchObservedRunningTime="2026-07-23 16:23:23.04290018 +0000 UTC m=+2066.247267573"
Jul 23 16:23:32.709063 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:32.709019    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:32.709486 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:32.709087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:32.721402 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:32.721370    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:33.086171 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:33.086082    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:23:49.156154 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:49.156108    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"]
Jul 23 16:23:49.162594 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:49.162569    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"
Jul 23 16:23:49.163379 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:49.162445    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"]
Jul 23 16:23:49.166139 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:49.166115    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w6jwk\""
Jul 23 16:23:49.264441 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:49.264406    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ccd613-b599-4689-bb4d-e5ffd14dd617-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2\" (UID: \"53ccd613-b599-4689-bb4d-e5ffd14dd617\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"
Jul 23 16:23:49.365412 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:49.365375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ccd613-b599-4689-bb4d-e5ffd14dd617-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2\" (UID: \"53ccd613-b599-4689-bb4d-e5ffd14dd617\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"
Jul 23 16:23:49.366125 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:49.366098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ccd613-b599-4689-bb4d-e5ffd14dd617-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2\" (UID: \"53ccd613-b599-4689-bb4d-e5ffd14dd617\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"
Jul 23 16:23:49.478161 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:49.478126    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"
Jul 23 16:23:49.826907 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:49.826839    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"]
Jul 23 16:23:49.829392 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:23:49.829356    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ccd613_b599_4689_bb4d_e5ffd14dd617.slice/crio-a2864cc6acac8b9ddd3c8847d67431cd84abb3f40dd01ef7662d7d8c61185595 WatchSource:0}: Error finding container a2864cc6acac8b9ddd3c8847d67431cd84abb3f40dd01ef7662d7d8c61185595: Status 404 returned error can't find the container with id a2864cc6acac8b9ddd3c8847d67431cd84abb3f40dd01ef7662d7d8c61185595
Jul 23 16:23:50.174423 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:50.174385    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2" event={"ID":"53ccd613-b599-4689-bb4d-e5ffd14dd617","Type":"ContainerStarted","Data":"b3707b0bc1720d2d3e117d0aede6bac55a27fe703974f2ed1707470737e8611d"}
Jul 23 16:23:50.174946 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:50.174439    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2" event={"ID":"53ccd613-b599-4689-bb4d-e5ffd14dd617","Type":"ContainerStarted","Data":"a2864cc6acac8b9ddd3c8847d67431cd84abb3f40dd01ef7662d7d8c61185595"}
Jul 23 16:23:50.194077 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:50.194022    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2" podStartSLOduration=1.1940045559999999 podStartE2EDuration="1.194004556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:23:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:23:50.192392117 +0000 UTC m=+2093.396759511" watchObservedRunningTime="2026-07-23 16:23:50.194004556 +0000 UTC m=+2093.398371948"
Jul 23 16:23:51.181692 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:51.181654    2562 generic.go:356] "Generic (PLEG): container finished" podID="53ccd613-b599-4689-bb4d-e5ffd14dd617" containerID="b3707b0bc1720d2d3e117d0aede6bac55a27fe703974f2ed1707470737e8611d" exitCode=0
Jul 23 16:23:51.182123 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:51.181743    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2" event={"ID":"53ccd613-b599-4689-bb4d-e5ffd14dd617","Type":"ContainerDied","Data":"b3707b0bc1720d2d3e117d0aede6bac55a27fe703974f2ed1707470737e8611d"}
Jul 23 16:23:52.326338 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:52.326314    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"
Jul 23 16:23:52.395662 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:52.395634    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ccd613-b599-4689-bb4d-e5ffd14dd617-maas-api-service-ca\") pod \"53ccd613-b599-4689-bb4d-e5ffd14dd617\" (UID: \"53ccd613-b599-4689-bb4d-e5ffd14dd617\") "
Jul 23 16:23:52.396124 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:52.396091    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53ccd613-b599-4689-bb4d-e5ffd14dd617-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "53ccd613-b599-4689-bb4d-e5ffd14dd617" (UID: "53ccd613-b599-4689-bb4d-e5ffd14dd617"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:23:52.497270 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:52.497183    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ccd613-b599-4689-bb4d-e5ffd14dd617-maas-api-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:23:53.193910 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:53.193879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2" event={"ID":"53ccd613-b599-4689-bb4d-e5ffd14dd617","Type":"ContainerDied","Data":"a2864cc6acac8b9ddd3c8847d67431cd84abb3f40dd01ef7662d7d8c61185595"}
Jul 23 16:23:53.193910 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:53.193903    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"
Jul 23 16:23:53.194143 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:53.193925    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2864cc6acac8b9ddd3c8847d67431cd84abb3f40dd01ef7662d7d8c61185595"
Jul 23 16:23:57.314106 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:57.314077    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:23:57.315362 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:57.315335    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:23:57.408283 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:23:57.408241    2562 scope.go:117] "RemoveContainer" containerID="56f4c1328420f7814e9d3fd1797d36592759b8b2e8c87fb6b213fbe191d6614f"
Jul 23 16:24:04.630583 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:04.630538    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"]
Jul 23 16:24:04.631217 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:04.631182    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm" podUID="6cf8dbab-d242-4680-b9b6-95f33dac9896" containerName="maas-api" containerID="cri-o://b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887" gracePeriod=30
Jul 23 16:24:04.882372 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:04.882309    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:24:05.028504 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.028463    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cf8dbab-d242-4680-b9b6-95f33dac9896-openshift-service-ca\") pod \"6cf8dbab-d242-4680-b9b6-95f33dac9896\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") "
Jul 23 16:24:05.028701 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.028533    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8q6p\" (UniqueName: \"kubernetes.io/projected/6cf8dbab-d242-4680-b9b6-95f33dac9896-kube-api-access-z8q6p\") pod \"6cf8dbab-d242-4680-b9b6-95f33dac9896\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") "
Jul 23 16:24:05.028701 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.028593    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cf8dbab-d242-4680-b9b6-95f33dac9896-maas-api-tls\") pod \"6cf8dbab-d242-4680-b9b6-95f33dac9896\" (UID: \"6cf8dbab-d242-4680-b9b6-95f33dac9896\") "
Jul 23 16:24:05.028896 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.028873    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cf8dbab-d242-4680-b9b6-95f33dac9896-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6cf8dbab-d242-4680-b9b6-95f33dac9896" (UID: "6cf8dbab-d242-4680-b9b6-95f33dac9896"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:24:05.028985 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.028957    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cf8dbab-d242-4680-b9b6-95f33dac9896-openshift-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:24:05.030760 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.030736    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cf8dbab-d242-4680-b9b6-95f33dac9896-kube-api-access-z8q6p" (OuterVolumeSpecName: "kube-api-access-z8q6p") pod "6cf8dbab-d242-4680-b9b6-95f33dac9896" (UID: "6cf8dbab-d242-4680-b9b6-95f33dac9896"). InnerVolumeSpecName "kube-api-access-z8q6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:24:05.030760 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.030742    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cf8dbab-d242-4680-b9b6-95f33dac9896-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6cf8dbab-d242-4680-b9b6-95f33dac9896" (UID: "6cf8dbab-d242-4680-b9b6-95f33dac9896"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:24:05.129894 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.129854    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8q6p\" (UniqueName: \"kubernetes.io/projected/6cf8dbab-d242-4680-b9b6-95f33dac9896-kube-api-access-z8q6p\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:24:05.129894 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.129887    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cf8dbab-d242-4680-b9b6-95f33dac9896-maas-api-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:24:05.257867 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.257827    2562 generic.go:356] "Generic (PLEG): container finished" podID="6cf8dbab-d242-4680-b9b6-95f33dac9896" containerID="b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887" exitCode=0
Jul 23 16:24:05.257867 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.257868    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"
Jul 23 16:24:05.258155 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.257897    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm" event={"ID":"6cf8dbab-d242-4680-b9b6-95f33dac9896","Type":"ContainerDied","Data":"b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887"}
Jul 23 16:24:05.258155 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.257943    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm" event={"ID":"6cf8dbab-d242-4680-b9b6-95f33dac9896","Type":"ContainerDied","Data":"3b263572d2a7aea55d769069e3f3bbfb956ad903a79a26c5063f785d8e875822"}
Jul 23 16:24:05.258155 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.257965    2562 scope.go:117] "RemoveContainer" containerID="b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887"
Jul 23 16:24:05.268910 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.268892    2562 scope.go:117] "RemoveContainer" containerID="b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887"
Jul 23 16:24:05.269169 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:24:05.269151    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887\": container with ID starting with b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887 not found: ID does not exist" containerID="b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887"
Jul 23 16:24:05.269217 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.269176    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887"} err="failed to get container status \"b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887\": rpc error: code = NotFound desc = could not find container \"b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887\": container with ID starting with b5b9f1c4ef0008ed2341aa83c36747e8ee2c217c700c8b9c5a1ed664af522887 not found: ID does not exist"
Jul 23 16:24:05.280058 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.280033    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"]
Jul 23 16:24:05.284830 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:05.284789    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b72f23f-868d9c89cd-gm6fm"]
Jul 23 16:24:07.255088 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:07.255056    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cf8dbab-d242-4680-b9b6-95f33dac9896" path="/var/lib/kubelet/pods/6cf8dbab-d242-4680-b9b6-95f33dac9896/volumes"
Jul 23 16:24:09.223626 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:09.223590    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"]
Jul 23 16:24:09.226269 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:09.226244    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4b72f23f-dl2m2"]
Jul 23 16:24:09.254792 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:09.254747    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53ccd613-b599-4689-bb4d-e5ffd14dd617" path="/var/lib/kubelet/pods/53ccd613-b599-4689-bb4d-e5ffd14dd617/volumes"
Jul 23 16:24:16.650549 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:16.650509    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"]
Jul 23 16:24:31.671663 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.671619    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5" podUID="80685dc9-6134-4a70-aa03-fc767be729fc" containerName="main" containerID="cri-o://cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d" gracePeriod=45
Jul 23 16:24:31.926568 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.926500    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:24:31.985484 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.985454    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrmfr\" (UniqueName: \"kubernetes.io/projected/80685dc9-6134-4a70-aa03-fc767be729fc-kube-api-access-zrmfr\") pod \"80685dc9-6134-4a70-aa03-fc767be729fc\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") "
Jul 23 16:24:31.985673 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.985491    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-model-cache\") pod \"80685dc9-6134-4a70-aa03-fc767be729fc\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") "
Jul 23 16:24:31.985673 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.985531    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-dshm\") pod \"80685dc9-6134-4a70-aa03-fc767be729fc\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") "
Jul 23 16:24:31.985673 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.985576    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-tmp-dir\") pod \"80685dc9-6134-4a70-aa03-fc767be729fc\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") "
Jul 23 16:24:31.985673 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.985605    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80685dc9-6134-4a70-aa03-fc767be729fc-tls-certs\") pod \"80685dc9-6134-4a70-aa03-fc767be729fc\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") "
Jul 23 16:24:31.985673 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.985636    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-home\") pod \"80685dc9-6134-4a70-aa03-fc767be729fc\" (UID: \"80685dc9-6134-4a70-aa03-fc767be729fc\") "
Jul 23 16:24:31.985981 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.985910    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "80685dc9-6134-4a70-aa03-fc767be729fc" (UID: "80685dc9-6134-4a70-aa03-fc767be729fc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:24:31.985981 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.985910    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-model-cache" (OuterVolumeSpecName: "model-cache") pod "80685dc9-6134-4a70-aa03-fc767be729fc" (UID: "80685dc9-6134-4a70-aa03-fc767be729fc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:24:31.985981 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.985961    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-home" (OuterVolumeSpecName: "home") pod "80685dc9-6134-4a70-aa03-fc767be729fc" (UID: "80685dc9-6134-4a70-aa03-fc767be729fc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:24:31.986145 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.986082    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-home\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:24:31.986145 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.986103    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-model-cache\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:24:31.986145 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.986119    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-tmp-dir\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:24:31.988049 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.988019    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80685dc9-6134-4a70-aa03-fc767be729fc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "80685dc9-6134-4a70-aa03-fc767be729fc" (UID: "80685dc9-6134-4a70-aa03-fc767be729fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:24:31.988232 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.988209    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-dshm" (OuterVolumeSpecName: "dshm") pod "80685dc9-6134-4a70-aa03-fc767be729fc" (UID: "80685dc9-6134-4a70-aa03-fc767be729fc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:24:31.988287 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:31.988229    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80685dc9-6134-4a70-aa03-fc767be729fc-kube-api-access-zrmfr" (OuterVolumeSpecName: "kube-api-access-zrmfr") pod "80685dc9-6134-4a70-aa03-fc767be729fc" (UID: "80685dc9-6134-4a70-aa03-fc767be729fc"). InnerVolumeSpecName "kube-api-access-zrmfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:24:32.086789 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.086756    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrmfr\" (UniqueName: \"kubernetes.io/projected/80685dc9-6134-4a70-aa03-fc767be729fc-kube-api-access-zrmfr\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:24:32.086789 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.086782    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80685dc9-6134-4a70-aa03-fc767be729fc-dshm\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:24:32.086789 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.086793    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80685dc9-6134-4a70-aa03-fc767be729fc-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:24:32.405820 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.405718    2562 generic.go:356] "Generic (PLEG): container finished" podID="80685dc9-6134-4a70-aa03-fc767be729fc" containerID="cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d" exitCode=0
Jul 23 16:24:32.405820 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.405763    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"
Jul 23 16:24:32.406012 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.405826    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5" event={"ID":"80685dc9-6134-4a70-aa03-fc767be729fc","Type":"ContainerDied","Data":"cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d"}
Jul 23 16:24:32.406012 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.405865    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5" event={"ID":"80685dc9-6134-4a70-aa03-fc767be729fc","Type":"ContainerDied","Data":"1fd3a5be96c9b95185fbab97b998cc554f2f662a3930fcf07de70b53b18b4808"}
Jul 23 16:24:32.406012 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.405889    2562 scope.go:117] "RemoveContainer" containerID="cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d"
Jul 23 16:24:32.415113 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.415091    2562 scope.go:117] "RemoveContainer" containerID="cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d"
Jul 23 16:24:32.415397 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:24:32.415379    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d\": container with ID starting with cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d not found: ID does not exist" containerID="cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d"
Jul 23 16:24:32.415447 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.415408    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d"} err="failed to get container status \"cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d\": rpc error: code = NotFound desc = could not find container \"cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d\": container with ID starting with cf9f6e553711325492d26044d536114022698ba3ebdc4a5217e6aaf69326232d not found: ID does not exist"
Jul 23 16:24:32.426604 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.426579    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"]
Jul 23 16:24:32.428405 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:32.428385    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-4b72f23f/e2e-lifecycle-model-4b72f23f-kserve-58db77595-q2gc5"]
Jul 23 16:24:33.254553 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:33.254515    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80685dc9-6134-4a70-aa03-fc767be729fc" path="/var/lib/kubelet/pods/80685dc9-6134-4a70-aa03-fc767be729fc/volumes"
Jul 23 16:24:57.430901 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:24:57.430796    2562 scope.go:117] "RemoveContainer" containerID="43a36673a99d162870b8cc39b80193abd4a1a097eca13ab28d21475c4dfc497c"
Jul 23 16:25:43.662106 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.662069    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"]
Jul 23 16:25:43.662617 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.662454    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80685dc9-6134-4a70-aa03-fc767be729fc" containerName="main"
Jul 23 16:25:43.662617 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.662464    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="80685dc9-6134-4a70-aa03-fc767be729fc" containerName="main"
Jul 23 16:25:43.662617 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.662487    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cf8dbab-d242-4680-b9b6-95f33dac9896" containerName="maas-api"
Jul 23 16:25:43.662617 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.662493    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cf8dbab-d242-4680-b9b6-95f33dac9896" containerName="maas-api"
Jul 23 16:25:43.662617 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.662507    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53ccd613-b599-4689-bb4d-e5ffd14dd617" containerName="revoke-keys"
Jul 23 16:25:43.662617 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.662512    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="53ccd613-b599-4689-bb4d-e5ffd14dd617" containerName="revoke-keys"
Jul 23 16:25:43.662617 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.662573    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="80685dc9-6134-4a70-aa03-fc767be729fc" containerName="main"
Jul 23 16:25:43.662617 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.662584    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="53ccd613-b599-4689-bb4d-e5ffd14dd617" containerName="revoke-keys"
Jul 23 16:25:43.662617 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.662592    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cf8dbab-d242-4680-b9b6-95f33dac9896" containerName="maas-api"
Jul 23 16:25:43.665722 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.665703    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:43.668631 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.668608    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5j8mj\""
Jul 23 16:25:43.668768 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.668665    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e51bfc-serving-cert\""
Jul 23 16:25:43.669868 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.669851    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 16:25:43.669961 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.669882    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 16:25:43.681817 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.681781    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"]
Jul 23 16:25:43.703134 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.703087    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/579902a7-2be0-4c05-9252-997d04e06e3b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:43.703134 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.703131    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579902a7-2be0-4c05-9252-997d04e06e3b-maas-api-tls\") pod \"maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:43.703414 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.703208    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lvmq\" (UniqueName: \"kubernetes.io/projected/579902a7-2be0-4c05-9252-997d04e06e3b-kube-api-access-2lvmq\") pod \"maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:43.804410 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.804377    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/579902a7-2be0-4c05-9252-997d04e06e3b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:43.804410 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.804416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579902a7-2be0-4c05-9252-997d04e06e3b-maas-api-tls\") pod \"maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:43.804684 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.804489    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lvmq\" (UniqueName: \"kubernetes.io/projected/579902a7-2be0-4c05-9252-997d04e06e3b-kube-api-access-2lvmq\") pod \"maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:43.804684 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:25:43.804620    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-serving-cert: secret "maas-api-e2e-inf-a-e51bfc-serving-cert" not found
Jul 23 16:25:43.804825 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:25:43.804700    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/579902a7-2be0-4c05-9252-997d04e06e3b-maas-api-tls podName:579902a7-2be0-4c05-9252-997d04e06e3b nodeName:}" failed. No retries permitted until 2026-07-23 16:25:44.304679107 +0000 UTC m=+2207.509046492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/579902a7-2be0-4c05-9252-997d04e06e3b-maas-api-tls") pod "maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7" (UID: "579902a7-2be0-4c05-9252-997d04e06e3b") : secret "maas-api-e2e-inf-a-e51bfc-serving-cert" not found
Jul 23 16:25:43.805148 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.805123    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/579902a7-2be0-4c05-9252-997d04e06e3b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:43.814366 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:43.814340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lvmq\" (UniqueName: \"kubernetes.io/projected/579902a7-2be0-4c05-9252-997d04e06e3b-kube-api-access-2lvmq\") pod \"maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:44.310251 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:44.310214    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579902a7-2be0-4c05-9252-997d04e06e3b-maas-api-tls\") pod \"maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:44.312675 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:44.312654    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579902a7-2be0-4c05-9252-997d04e06e3b-maas-api-tls\") pod \"maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:44.576974 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:44.576880    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:44.719290 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:44.719253    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"]
Jul 23 16:25:44.720559 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:25:44.720517    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579902a7_2be0_4c05_9252_997d04e06e3b.slice/crio-b52d2b8396af9b620e72114573f94cd717123e524af3e8665b5b9e9be4f5ae71 WatchSource:0}: Error finding container b52d2b8396af9b620e72114573f94cd717123e524af3e8665b5b9e9be4f5ae71: Status 404 returned error can't find the container with id b52d2b8396af9b620e72114573f94cd717123e524af3e8665b5b9e9be4f5ae71
Jul 23 16:25:44.795996 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:44.795962    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7" event={"ID":"579902a7-2be0-4c05-9252-997d04e06e3b","Type":"ContainerStarted","Data":"b52d2b8396af9b620e72114573f94cd717123e524af3e8665b5b9e9be4f5ae71"}
Jul 23 16:25:45.803672 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:45.803634    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7" event={"ID":"579902a7-2be0-4c05-9252-997d04e06e3b","Type":"ContainerStarted","Data":"cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33"}
Jul 23 16:25:45.804197 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:45.803732    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:45.822955 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:45.822900    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7" podStartSLOduration=2.364368072 podStartE2EDuration="2.822880277s" totalImagesPullingTime="458.512205ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:25:45.821076994 +0000 UTC m=+2209.025444411" watchObservedRunningTime="2026-07-23 16:25:45.822880277 +0000 UTC m=+2209.027247669"
Jul 23 16:25:48.683639 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.683598    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"]
Jul 23 16:25:48.687839 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.687813    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.690657 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.690634    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 16:25:48.690828 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.690662    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-799a66-openshift-default-dockercfg-m5mzt\""
Jul 23 16:25:48.697393 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.697364    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"]
Jul 23 16:25:48.755454 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.755418    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzkdq\" (UniqueName: \"kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-kube-api-access-dzkdq\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.755667 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.755482    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-credential-socket\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.755667 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.755558    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-token\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.755667 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.755579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-envoy\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.755667 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.755600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-socket\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.755667 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.755637    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-data\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.755969 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.755700    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istiod-ca-cert\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.755969 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.755770    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-certs\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.755969 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.755857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-podinfo\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857153 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzkdq\" (UniqueName: \"kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-kube-api-access-dzkdq\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857353 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857193    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-credential-socket\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857353 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857236    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-token\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857353 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857265    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-envoy\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857353 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857299    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-socket\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857353 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857353    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-data\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857643 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857377    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istiod-ca-cert\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857643 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-certs\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857643 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-podinfo\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857830 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857650    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-credential-socket\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857830 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857689    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-socket\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.857999 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857980    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-data\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.858079 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.857977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-certs\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.858209 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.858187    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istiod-ca-cert\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.859840 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.859817    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-podinfo\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.859998 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.859976    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-envoy\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.867330 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.867303    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-token\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.867484 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.867463    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzkdq\" (UniqueName: \"kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-kube-api-access-dzkdq\") pod \"e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") " pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:48.999724 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:48.999636    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:49.148228 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:49.148202    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"]
Jul 23 16:25:49.149608 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:25:49.149577    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5009bef_47da_44a5_9623_9b12cf9eb9e8.slice/crio-f5b92c44f3116a836ee10c6f509df07c43d9267a5265a238ed1ad0645c5a1b0f WatchSource:0}: Error finding container f5b92c44f3116a836ee10c6f509df07c43d9267a5265a238ed1ad0645c5a1b0f: Status 404 returned error can't find the container with id f5b92c44f3116a836ee10c6f509df07c43d9267a5265a238ed1ad0645c5a1b0f
Jul 23 16:25:49.829701 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:49.829669    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" event={"ID":"e5009bef-47da-44a5-9623-9b12cf9eb9e8","Type":"ContainerStarted","Data":"f5b92c44f3116a836ee10c6f509df07c43d9267a5265a238ed1ad0645c5a1b0f"}
Jul 23 16:25:51.815820 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:51.815770    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:25:52.010975 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:52.010941    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:25:52.011062 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:52.011010    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:25:52.011062 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:52.011040    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:25:52.853246 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:52.853203    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" event={"ID":"e5009bef-47da-44a5-9623-9b12cf9eb9e8","Type":"ContainerStarted","Data":"49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba"}
Jul 23 16:25:52.876015 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:52.875962    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" podStartSLOduration=2.017201592 podStartE2EDuration="4.87594347s" totalImagesPullingTime="2.858741878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:25:52.874834024 +0000 UTC m=+2216.079201408" watchObservedRunningTime="2026-07-23 16:25:52.87594347 +0000 UTC m=+2216.080310863"
Jul 23 16:25:52.999777 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:52.999740    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:54.005690 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:54.005658    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:54.865875 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:54.865839    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:54.866907 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:54.866884    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:25:55.108149 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.108119    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"]
Jul 23 16:25:55.111864 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.111844    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:55.114641 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.114616    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-799a66-serving-cert\""
Jul 23 16:25:55.121819 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.121774    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"]
Jul 23 16:25:55.230060 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.230021    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84x4t\" (UniqueName: \"kubernetes.io/projected/4f67125e-518c-4374-b67d-900adb902e50-kube-api-access-84x4t\") pod \"maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:55.230259 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.230150    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f67125e-518c-4374-b67d-900adb902e50-openshift-service-ca\") pod \"maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:55.230259 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.230180    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f67125e-518c-4374-b67d-900adb902e50-maas-api-tls\") pod \"maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:55.330975 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.330941    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84x4t\" (UniqueName: \"kubernetes.io/projected/4f67125e-518c-4374-b67d-900adb902e50-kube-api-access-84x4t\") pod \"maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:55.331181 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.331043    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f67125e-518c-4374-b67d-900adb902e50-openshift-service-ca\") pod \"maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:55.331181 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.331063    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f67125e-518c-4374-b67d-900adb902e50-maas-api-tls\") pod \"maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:55.331765 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.331739    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f67125e-518c-4374-b67d-900adb902e50-openshift-service-ca\") pod \"maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:55.333517 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.333495    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f67125e-518c-4374-b67d-900adb902e50-maas-api-tls\") pod \"maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:55.339930 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.339909    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84x4t\" (UniqueName: \"kubernetes.io/projected/4f67125e-518c-4374-b67d-900adb902e50-kube-api-access-84x4t\") pod \"maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:55.424056 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.424017    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:55.588001 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.587968    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"]
Jul 23 16:25:55.589417 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:25:55.589387    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f67125e_518c_4374_b67d_900adb902e50.slice/crio-b261d3fb051d650194183217b1c983e7ef29794a7fd80407293188db19db92ef WatchSource:0}: Error finding container b261d3fb051d650194183217b1c983e7ef29794a7fd80407293188db19db92ef: Status 404 returned error can't find the container with id b261d3fb051d650194183217b1c983e7ef29794a7fd80407293188db19db92ef
Jul 23 16:25:55.872024 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:55.871929    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8" event={"ID":"4f67125e-518c-4374-b67d-900adb902e50","Type":"ContainerStarted","Data":"b261d3fb051d650194183217b1c983e7ef29794a7fd80407293188db19db92ef"}
Jul 23 16:25:56.882015 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:56.881975    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8" event={"ID":"4f67125e-518c-4374-b67d-900adb902e50","Type":"ContainerStarted","Data":"45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49"}
Jul 23 16:25:56.882514 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:56.882268    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:25:56.900876 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:56.900829    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8" podStartSLOduration=1.5785822280000001 podStartE2EDuration="1.900812113s" totalImagesPullingTime="322.229885ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:25:56.899887294 +0000 UTC m=+2220.104254690" watchObservedRunningTime="2026-07-23 16:25:56.900812113 +0000 UTC m=+2220.105179497"
Jul 23 16:25:57.461821 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:25:57.461781    2562 scope.go:117] "RemoveContainer" containerID="bae557a2b9dddcc60bae8b26efe26289703010e64adae66c8f638a8811dda4e3"
Jul 23 16:26:02.894638 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:02.894609    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:26:05.718238 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.718204    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"]
Jul 23 16:26:05.722181 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.722163    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.725054 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.725028    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e51bfc\"/\"openshift-service-ca.crt\""
Jul 23 16:26:05.726417 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.726391    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e51bfc\"/\"kube-root-ca.crt\""
Jul 23 16:26:05.726555 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.726394    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e51bfc\"/\"test-model-e51bfc-kserve-self-signed-certs\""
Jul 23 16:26:05.726555 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.726406    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e51bfc\"/\"default-dockercfg-6khsz\""
Jul 23 16:26:05.732149 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.732122    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"]
Jul 23 16:26:05.732349 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.732328    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-model-cache\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.732458 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.732374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-dshm\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.732458 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.732411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hwk9\" (UniqueName: \"kubernetes.io/projected/8426cd7f-fb32-4af3-bfc4-4ffae6741447-kube-api-access-9hwk9\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.732598 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.732511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tmp-dir\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.732598 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.732557    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tls-certs\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.732711 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.732648    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-home\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.833486 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.833451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-home\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.833486 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.833494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-model-cache\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.833761 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.833522    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-dshm\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.833761 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.833542    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hwk9\" (UniqueName: \"kubernetes.io/projected/8426cd7f-fb32-4af3-bfc4-4ffae6741447-kube-api-access-9hwk9\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.833761 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.833596    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tmp-dir\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.833761 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.833689    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tls-certs\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.834024 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.833961    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-home\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.834024 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.833994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-model-cache\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.834104 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.834084    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tmp-dir\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.836075 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.836047    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-dshm\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.836327 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.836304    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tls-certs\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:05.845129 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:05.845093    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hwk9\" (UniqueName: \"kubernetes.io/projected/8426cd7f-fb32-4af3-bfc4-4ffae6741447-kube-api-access-9hwk9\") pod \"test-model-e51bfc-kserve-6977d59689-wtchg\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") " pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:06.035514 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:06.035413    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:06.201657 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:06.201559    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"]
Jul 23 16:26:06.204368 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:26:06.204339    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8426cd7f_fb32_4af3_bfc4_4ffae6741447.slice/crio-46267b9a29f0ad635c5c7757a3220f2f0ab546a85efb2d42675ecbede9e9af37 WatchSource:0}: Error finding container 46267b9a29f0ad635c5c7757a3220f2f0ab546a85efb2d42675ecbede9e9af37: Status 404 returned error can't find the container with id 46267b9a29f0ad635c5c7757a3220f2f0ab546a85efb2d42675ecbede9e9af37
Jul 23 16:26:06.942705 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:06.942670    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg" event={"ID":"8426cd7f-fb32-4af3-bfc4-4ffae6741447","Type":"ContainerStarted","Data":"e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078"}
Jul 23 16:26:06.943103 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:06.942722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg" event={"ID":"8426cd7f-fb32-4af3-bfc4-4ffae6741447","Type":"ContainerStarted","Data":"46267b9a29f0ad635c5c7757a3220f2f0ab546a85efb2d42675ecbede9e9af37"}
Jul 23 16:26:06.962530 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:06.962481    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg" podStartSLOduration=1.962460952 podStartE2EDuration="1.962460952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:26:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:26:06.960028632 +0000 UTC m=+2230.164396030" watchObservedRunningTime="2026-07-23 16:26:06.962460952 +0000 UTC m=+2230.166828346"
Jul 23 16:26:07.950111 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:07.950076    2562 generic.go:356] "Generic (PLEG): container finished" podID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerID="49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba" exitCode=0
Jul 23 16:26:07.950605 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:07.950163    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" event={"ID":"e5009bef-47da-44a5-9623-9b12cf9eb9e8","Type":"ContainerDied","Data":"49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba"}
Jul 23 16:26:07.950758 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:07.950739    2562 scope.go:117] "RemoveContainer" containerID="49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba"
Jul 23 16:26:07.951680 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:07.951656    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:26:07.951744 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:07.951730    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:26:07.951788 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:07.951766    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:26:08.959679 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:08.959645    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" event={"ID":"e5009bef-47da-44a5-9623-9b12cf9eb9e8","Type":"ContainerStarted","Data":"7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b"}
Jul 23 16:26:08.999776 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:08.999743    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:26:09.866022 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:09.865978    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:26:10.001226 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:10.001188    2562 patch_prober.go:28] interesting pod/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 23 16:26:10.001640 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:10.001264    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 23 16:26:11.000904 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:11.000861    2562 patch_prober.go:28] interesting pod/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 23 16:26:11.001097 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:11.000936    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 23 16:26:12.000134 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:12.000098    2562 patch_prober.go:28] interesting pod/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 23 16:26:12.000527 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:12.000155    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 23 16:26:13.000580 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:13.000539    2562 patch_prober.go:28] interesting pod/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 23 16:26:13.000991 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:13.000606    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 23 16:26:14.004196 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:14.004166    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:26:14.005030 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:14.005011    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:26:16.036175 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:16.036074    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:16.036175 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:16.036150    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:16.049535 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:16.049505    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:17.024305 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:17.024279    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:26:57.484240 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:26:57.484165    2562 scope.go:117] "RemoveContainer" containerID="a07a624fc99e29de7f1eaa0addbf8f8adc64c06262559a28b07ec661e90f64bd"
Jul 23 16:27:20.119293 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:20.119254    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"]
Jul 23 16:27:31.612098 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:31.612061    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"]
Jul 23 16:27:31.612738 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:31.612337    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7" podUID="579902a7-2be0-4c05-9252-997d04e06e3b" containerName="maas-api" containerID="cri-o://cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33" gracePeriod=30
Jul 23 16:27:31.866871 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:31.866796    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:27:32.019675 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.019640    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lvmq\" (UniqueName: \"kubernetes.io/projected/579902a7-2be0-4c05-9252-997d04e06e3b-kube-api-access-2lvmq\") pod \"579902a7-2be0-4c05-9252-997d04e06e3b\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") "
Jul 23 16:27:32.019902 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.019733    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579902a7-2be0-4c05-9252-997d04e06e3b-maas-api-tls\") pod \"579902a7-2be0-4c05-9252-997d04e06e3b\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") "
Jul 23 16:27:32.019902 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.019870    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/579902a7-2be0-4c05-9252-997d04e06e3b-openshift-service-ca\") pod \"579902a7-2be0-4c05-9252-997d04e06e3b\" (UID: \"579902a7-2be0-4c05-9252-997d04e06e3b\") "
Jul 23 16:27:32.020178 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.020151    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/579902a7-2be0-4c05-9252-997d04e06e3b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "579902a7-2be0-4c05-9252-997d04e06e3b" (UID: "579902a7-2be0-4c05-9252-997d04e06e3b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:27:32.020268 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.020249    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/579902a7-2be0-4c05-9252-997d04e06e3b-openshift-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:32.021934 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.021907    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/579902a7-2be0-4c05-9252-997d04e06e3b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "579902a7-2be0-4c05-9252-997d04e06e3b" (UID: "579902a7-2be0-4c05-9252-997d04e06e3b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:27:32.022046 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.021907    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/579902a7-2be0-4c05-9252-997d04e06e3b-kube-api-access-2lvmq" (OuterVolumeSpecName: "kube-api-access-2lvmq") pod "579902a7-2be0-4c05-9252-997d04e06e3b" (UID: "579902a7-2be0-4c05-9252-997d04e06e3b"). InnerVolumeSpecName "kube-api-access-2lvmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:27:32.121580 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.121488    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lvmq\" (UniqueName: \"kubernetes.io/projected/579902a7-2be0-4c05-9252-997d04e06e3b-kube-api-access-2lvmq\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:32.121580 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.121522    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579902a7-2be0-4c05-9252-997d04e06e3b-maas-api-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:32.419618 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.419585    2562 generic.go:356] "Generic (PLEG): container finished" podID="579902a7-2be0-4c05-9252-997d04e06e3b" containerID="cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33" exitCode=0
Jul 23 16:27:32.419851 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.419633    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"
Jul 23 16:27:32.419851 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.419641    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7" event={"ID":"579902a7-2be0-4c05-9252-997d04e06e3b","Type":"ContainerDied","Data":"cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33"}
Jul 23 16:27:32.419851 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.419700    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7" event={"ID":"579902a7-2be0-4c05-9252-997d04e06e3b","Type":"ContainerDied","Data":"b52d2b8396af9b620e72114573f94cd717123e524af3e8665b5b9e9be4f5ae71"}
Jul 23 16:27:32.419851 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.419730    2562 scope.go:117] "RemoveContainer" containerID="cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33"
Jul 23 16:27:32.430126 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.430106    2562 scope.go:117] "RemoveContainer" containerID="cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33"
Jul 23 16:27:32.430407 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:27:32.430387    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33\": container with ID starting with cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33 not found: ID does not exist" containerID="cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33"
Jul 23 16:27:32.430448 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.430417    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33"} err="failed to get container status \"cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33\": rpc error: code = NotFound desc = could not find container \"cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33\": container with ID starting with cc1e8b12bf01a1fa62b35af059071db2bc6ad26e53b7ae7a077617b139e32d33 not found: ID does not exist"
Jul 23 16:27:32.443010 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.442981    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"]
Jul 23 16:27:32.448348 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:32.448324    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7"]
Jul 23 16:27:33.254671 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:33.254624    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="579902a7-2be0-4c05-9252-997d04e06e3b" path="/var/lib/kubelet/pods/579902a7-2be0-4c05-9252-997d04e06e3b/volumes"
Jul 23 16:27:34.810559 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:34.810508    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e51bfc-5b5fc9c886-fvqh7" podUID="579902a7-2be0-4c05-9252-997d04e06e3b" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.56:8443/health\": context deadline exceeded"
Jul 23 16:27:35.140291 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.140204    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg" podUID="8426cd7f-fb32-4af3-bfc4-4ffae6741447" containerName="main" containerID="cri-o://e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078" gracePeriod=45
Jul 23 16:27:35.403074 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.403011    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:27:35.438139 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.438104    2562 generic.go:356] "Generic (PLEG): container finished" podID="8426cd7f-fb32-4af3-bfc4-4ffae6741447" containerID="e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078" exitCode=0
Jul 23 16:27:35.438139 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.438144    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"
Jul 23 16:27:35.438388 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.438174    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg" event={"ID":"8426cd7f-fb32-4af3-bfc4-4ffae6741447","Type":"ContainerDied","Data":"e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078"}
Jul 23 16:27:35.438388 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.438231    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg" event={"ID":"8426cd7f-fb32-4af3-bfc4-4ffae6741447","Type":"ContainerDied","Data":"46267b9a29f0ad635c5c7757a3220f2f0ab546a85efb2d42675ecbede9e9af37"}
Jul 23 16:27:35.438388 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.438259    2562 scope.go:117] "RemoveContainer" containerID="e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078"
Jul 23 16:27:35.449492 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.449468    2562 scope.go:117] "RemoveContainer" containerID="e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078"
Jul 23 16:27:35.449840 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:27:35.449815    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078\": container with ID starting with e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078 not found: ID does not exist" containerID="e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078"
Jul 23 16:27:35.449963 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.449851    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078"} err="failed to get container status \"e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078\": rpc error: code = NotFound desc = could not find container \"e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078\": container with ID starting with e14b5173d36c8fc7a3b95c6c6591e3b537d9e2729ff31a4b9a2bfd8180db4078 not found: ID does not exist"
Jul 23 16:27:35.555945 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.555911    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tmp-dir\") pod \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") "
Jul 23 16:27:35.556165 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.555984    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-dshm\") pod \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") "
Jul 23 16:27:35.556165 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.556027    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hwk9\" (UniqueName: \"kubernetes.io/projected/8426cd7f-fb32-4af3-bfc4-4ffae6741447-kube-api-access-9hwk9\") pod \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") "
Jul 23 16:27:35.556165 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.556059    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-home\") pod \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") "
Jul 23 16:27:35.556165 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.556119    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-model-cache\") pod \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") "
Jul 23 16:27:35.556165 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.556141    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tls-certs\") pod \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\" (UID: \"8426cd7f-fb32-4af3-bfc4-4ffae6741447\") "
Jul 23 16:27:35.556462 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.556235    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8426cd7f-fb32-4af3-bfc4-4ffae6741447" (UID: "8426cd7f-fb32-4af3-bfc4-4ffae6741447"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:27:35.556462 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.556403    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-home" (OuterVolumeSpecName: "home") pod "8426cd7f-fb32-4af3-bfc4-4ffae6741447" (UID: "8426cd7f-fb32-4af3-bfc4-4ffae6741447"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:27:35.556577 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.556487    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tmp-dir\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:35.556577 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.556502    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-home\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:35.556725 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.556707    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-model-cache" (OuterVolumeSpecName: "model-cache") pod "8426cd7f-fb32-4af3-bfc4-4ffae6741447" (UID: "8426cd7f-fb32-4af3-bfc4-4ffae6741447"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:27:35.558396 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.558366    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8426cd7f-fb32-4af3-bfc4-4ffae6741447" (UID: "8426cd7f-fb32-4af3-bfc4-4ffae6741447"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:27:35.558571 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.558551    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-dshm" (OuterVolumeSpecName: "dshm") pod "8426cd7f-fb32-4af3-bfc4-4ffae6741447" (UID: "8426cd7f-fb32-4af3-bfc4-4ffae6741447"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:27:35.558697 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.558678    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8426cd7f-fb32-4af3-bfc4-4ffae6741447-kube-api-access-9hwk9" (OuterVolumeSpecName: "kube-api-access-9hwk9") pod "8426cd7f-fb32-4af3-bfc4-4ffae6741447" (UID: "8426cd7f-fb32-4af3-bfc4-4ffae6741447"). InnerVolumeSpecName "kube-api-access-9hwk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:27:35.657449 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.657354    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-dshm\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:35.657449 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.657392    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hwk9\" (UniqueName: \"kubernetes.io/projected/8426cd7f-fb32-4af3-bfc4-4ffae6741447-kube-api-access-9hwk9\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:35.657449 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.657410    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8426cd7f-fb32-4af3-bfc4-4ffae6741447-model-cache\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:35.657449 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.657423    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8426cd7f-fb32-4af3-bfc4-4ffae6741447-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:35.762162 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.762129    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"]
Jul 23 16:27:35.765224 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:35.765195    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e51bfc/test-model-e51bfc-kserve-6977d59689-wtchg"]
Jul 23 16:27:37.264639 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:37.264606    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8426cd7f-fb32-4af3-bfc4-4ffae6741447" path="/var/lib/kubelet/pods/8426cd7f-fb32-4af3-bfc4-4ffae6741447/volumes"
Jul 23 16:27:43.622771 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.622727    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"]
Jul 23 16:27:43.623291 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.623274    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="579902a7-2be0-4c05-9252-997d04e06e3b" containerName="maas-api"
Jul 23 16:27:43.623341 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.623293    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="579902a7-2be0-4c05-9252-997d04e06e3b" containerName="maas-api"
Jul 23 16:27:43.623341 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.623326    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8426cd7f-fb32-4af3-bfc4-4ffae6741447" containerName="main"
Jul 23 16:27:43.623341 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.623333    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8426cd7f-fb32-4af3-bfc4-4ffae6741447" containerName="main"
Jul 23 16:27:43.623444 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.623397    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8426cd7f-fb32-4af3-bfc4-4ffae6741447" containerName="main"
Jul 23 16:27:43.623444 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.623406    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="579902a7-2be0-4c05-9252-997d04e06e3b" containerName="maas-api"
Jul 23 16:27:43.627705 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.627688    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"
Jul 23 16:27:43.631031 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.631001    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w6jwk\""
Jul 23 16:27:43.633684 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.633664    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"]
Jul 23 16:27:43.737717 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.737677    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/628c935f-1824-4a05-991f-61c596c6304d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-799a66-xvb62\" (UID: \"628c935f-1824-4a05-991f-61c596c6304d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"
Jul 23 16:27:43.838488 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.838451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/628c935f-1824-4a05-991f-61c596c6304d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-799a66-xvb62\" (UID: \"628c935f-1824-4a05-991f-61c596c6304d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"
Jul 23 16:27:43.839222 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.839193    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/628c935f-1824-4a05-991f-61c596c6304d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-799a66-xvb62\" (UID: \"628c935f-1824-4a05-991f-61c596c6304d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"
Jul 23 16:27:43.939399 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:43.939366    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"
Jul 23 16:27:44.085005 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:44.084972    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"]
Jul 23 16:27:44.086901 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:27:44.086874    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628c935f_1824_4a05_991f_61c596c6304d.slice/crio-9629209834eea27b1210eb5d958ac1a26badc6879ee15f6e565346995ae94960 WatchSource:0}: Error finding container 9629209834eea27b1210eb5d958ac1a26badc6879ee15f6e565346995ae94960: Status 404 returned error can't find the container with id 9629209834eea27b1210eb5d958ac1a26badc6879ee15f6e565346995ae94960
Jul 23 16:27:44.491512 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:44.491470    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62" event={"ID":"628c935f-1824-4a05-991f-61c596c6304d","Type":"ContainerStarted","Data":"09b99d28b950d16d648a295a0dec497e1b0ce3b326ff77138d5c0fe48093b738"}
Jul 23 16:27:44.491690 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:44.491541    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62" event={"ID":"628c935f-1824-4a05-991f-61c596c6304d","Type":"ContainerStarted","Data":"9629209834eea27b1210eb5d958ac1a26badc6879ee15f6e565346995ae94960"}
Jul 23 16:27:44.508628 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:44.508566    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62" podStartSLOduration=1.508544253 podStartE2EDuration="1.508544253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:27:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:27:44.507887812 +0000 UTC m=+2327.712255207" watchObservedRunningTime="2026-07-23 16:27:44.508544253 +0000 UTC m=+2327.712911645"
Jul 23 16:27:45.498175 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:45.498141    2562 generic.go:356] "Generic (PLEG): container finished" podID="628c935f-1824-4a05-991f-61c596c6304d" containerID="09b99d28b950d16d648a295a0dec497e1b0ce3b326ff77138d5c0fe48093b738" exitCode=0
Jul 23 16:27:45.498562 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:45.498228    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62" event={"ID":"628c935f-1824-4a05-991f-61c596c6304d","Type":"ContainerDied","Data":"09b99d28b950d16d648a295a0dec497e1b0ce3b326ff77138d5c0fe48093b738"}
Jul 23 16:27:46.642056 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:46.642031    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"
Jul 23 16:27:46.660322 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:46.660292    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/628c935f-1824-4a05-991f-61c596c6304d-maas-api-service-ca\") pod \"628c935f-1824-4a05-991f-61c596c6304d\" (UID: \"628c935f-1824-4a05-991f-61c596c6304d\") "
Jul 23 16:27:46.660758 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:46.660718    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/628c935f-1824-4a05-991f-61c596c6304d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "628c935f-1824-4a05-991f-61c596c6304d" (UID: "628c935f-1824-4a05-991f-61c596c6304d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:27:46.761249 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:46.761167    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/628c935f-1824-4a05-991f-61c596c6304d-maas-api-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:47.510834 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:47.510783    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62" event={"ID":"628c935f-1824-4a05-991f-61c596c6304d","Type":"ContainerDied","Data":"9629209834eea27b1210eb5d958ac1a26badc6879ee15f6e565346995ae94960"}
Jul 23 16:27:47.511005 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:47.510875    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9629209834eea27b1210eb5d958ac1a26badc6879ee15f6e565346995ae94960"
Jul 23 16:27:47.511005 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:47.510797    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"
Jul 23 16:27:54.154581 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.154496    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"]
Jul 23 16:27:54.155117 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.154868    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8" podUID="4f67125e-518c-4374-b67d-900adb902e50" containerName="maas-api" containerID="cri-o://45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49" gracePeriod=30
Jul 23 16:27:54.407401 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.407331    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:27:54.437666 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.437635    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f67125e-518c-4374-b67d-900adb902e50-openshift-service-ca\") pod \"4f67125e-518c-4374-b67d-900adb902e50\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") "
Jul 23 16:27:54.437861 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.437718    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84x4t\" (UniqueName: \"kubernetes.io/projected/4f67125e-518c-4374-b67d-900adb902e50-kube-api-access-84x4t\") pod \"4f67125e-518c-4374-b67d-900adb902e50\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") "
Jul 23 16:27:54.437861 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.437783    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f67125e-518c-4374-b67d-900adb902e50-maas-api-tls\") pod \"4f67125e-518c-4374-b67d-900adb902e50\" (UID: \"4f67125e-518c-4374-b67d-900adb902e50\") "
Jul 23 16:27:54.438035 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.438012    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f67125e-518c-4374-b67d-900adb902e50-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f67125e-518c-4374-b67d-900adb902e50" (UID: "4f67125e-518c-4374-b67d-900adb902e50"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:27:54.438188 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.438167    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f67125e-518c-4374-b67d-900adb902e50-openshift-service-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:54.440038 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.440008    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f67125e-518c-4374-b67d-900adb902e50-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f67125e-518c-4374-b67d-900adb902e50" (UID: "4f67125e-518c-4374-b67d-900adb902e50"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:27:54.440132 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.440035    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f67125e-518c-4374-b67d-900adb902e50-kube-api-access-84x4t" (OuterVolumeSpecName: "kube-api-access-84x4t") pod "4f67125e-518c-4374-b67d-900adb902e50" (UID: "4f67125e-518c-4374-b67d-900adb902e50"). InnerVolumeSpecName "kube-api-access-84x4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:27:54.539624 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.539590    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f67125e-518c-4374-b67d-900adb902e50-maas-api-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:54.539624 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.539618    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84x4t\" (UniqueName: \"kubernetes.io/projected/4f67125e-518c-4374-b67d-900adb902e50-kube-api-access-84x4t\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:27:54.549555 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.549522    2562 generic.go:356] "Generic (PLEG): container finished" podID="4f67125e-518c-4374-b67d-900adb902e50" containerID="45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49" exitCode=0
Jul 23 16:27:54.549742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.549575    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"
Jul 23 16:27:54.549742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.549604    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8" event={"ID":"4f67125e-518c-4374-b67d-900adb902e50","Type":"ContainerDied","Data":"45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49"}
Jul 23 16:27:54.549742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.549666    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8" event={"ID":"4f67125e-518c-4374-b67d-900adb902e50","Type":"ContainerDied","Data":"b261d3fb051d650194183217b1c983e7ef29794a7fd80407293188db19db92ef"}
Jul 23 16:27:54.549742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.549694    2562 scope.go:117] "RemoveContainer" containerID="45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49"
Jul 23 16:27:54.559729 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.559706    2562 scope.go:117] "RemoveContainer" containerID="45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49"
Jul 23 16:27:54.560089 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:27:54.560069    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49\": container with ID starting with 45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49 not found: ID does not exist" containerID="45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49"
Jul 23 16:27:54.560156 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.560100    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49"} err="failed to get container status \"45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49\": rpc error: code = NotFound desc = could not find container \"45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49\": container with ID starting with 45724b27728234b11c41625c0994d6536b645b6c45f54eccbc99aaaa5ae95b49 not found: ID does not exist"
Jul 23 16:27:54.574734 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.574702    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"]
Jul 23 16:27:54.579343 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:54.579315    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-799a66-76ff7bcf9c-2z9t8"]
Jul 23 16:27:55.253820 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:55.253783    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f67125e-518c-4374-b67d-900adb902e50" path="/var/lib/kubelet/pods/4f67125e-518c-4374-b67d-900adb902e50/volumes"
Jul 23 16:27:58.707352 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:58.707311    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"]
Jul 23 16:27:58.709209 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:58.709182    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-799a66-xvb62"]
Jul 23 16:27:59.254058 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:27:59.254022    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="628c935f-1824-4a05-991f-61c596c6304d" path="/var/lib/kubelet/pods/628c935f-1824-4a05-991f-61c596c6304d/volumes"
Jul 23 16:28:05.138894 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:05.138857    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"]
Jul 23 16:28:05.139447 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:05.139078    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerName="istio-proxy" containerID="cri-o://7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b" gracePeriod=30
Jul 23 16:28:06.077398 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.077360    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"]
Jul 23 16:28:06.077852 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.077794    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="628c935f-1824-4a05-991f-61c596c6304d" containerName="revoke-keys"
Jul 23 16:28:06.077852 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.077842    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="628c935f-1824-4a05-991f-61c596c6304d" containerName="revoke-keys"
Jul 23 16:28:06.078067 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.077871    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f67125e-518c-4374-b67d-900adb902e50" containerName="maas-api"
Jul 23 16:28:06.078067 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.077881    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f67125e-518c-4374-b67d-900adb902e50" containerName="maas-api"
Jul 23 16:28:06.078067 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.078028    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="628c935f-1824-4a05-991f-61c596c6304d" containerName="revoke-keys"
Jul 23 16:28:06.078067 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.078044    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f67125e-518c-4374-b67d-900adb902e50" containerName="maas-api"
Jul 23 16:28:06.081422 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.081398    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.084213 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.084186    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-89bfd8-openshift-default-dockercfg-zslps\""
Jul 23 16:28:06.094136 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.094109    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"]
Jul 23 16:28:06.154196 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.154163    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/595853a6-5319-4c62-a6e4-a08dd2598a88-istiod-ca-cert\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.154592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.154220    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-socket\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.154592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.154293    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-token\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.154592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.154357    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-credential-socket\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.154592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.154385    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-podinfo\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.154592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.154405    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9f5m\" (UniqueName: \"kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-kube-api-access-v9f5m\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.154592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.154484    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-data\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.154592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.154512    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-certs\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.154592 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.154550    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-envoy\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.255446 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.255411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-credential-socket\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.255446 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.255458    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-podinfo\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.255736 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.255479    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9f5m\" (UniqueName: \"kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-kube-api-access-v9f5m\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.255736 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.255516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-data\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.255736 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.255581    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-certs\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.255736 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.255639    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-envoy\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.255736 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.255668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/595853a6-5319-4c62-a6e4-a08dd2598a88-istiod-ca-cert\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.255736 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.255728    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-socket\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.256136 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.255772    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-token\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.256136 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.256025    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-credential-socket\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.256136 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.256049    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-data\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.256348 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.256330    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-socket\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.256441 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.256418    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-certs\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.256548 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.256528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/595853a6-5319-4c62-a6e4-a08dd2598a88-istiod-ca-cert\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.258258 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.258234    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-envoy\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.258496 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.258473    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-podinfo\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.264380 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.264347    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-token\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.264927 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.264900    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9f5m\" (UniqueName: \"kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-kube-api-access-v9f5m\") pod \"e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") " pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.392510 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.392405    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:06.540347 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.540320    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"]
Jul 23 16:28:06.542231 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:28:06.542202    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595853a6_5319_4c62_a6e4_a08dd2598a88.slice/crio-1fd19f597dc8893f08f99648f3caefbde7f1bf4a1faa4f8910d24fc92e94228c WatchSource:0}: Error finding container 1fd19f597dc8893f08f99648f3caefbde7f1bf4a1faa4f8910d24fc92e94228c: Status 404 returned error can't find the container with id 1fd19f597dc8893f08f99648f3caefbde7f1bf4a1faa4f8910d24fc92e94228c
Jul 23 16:28:06.544701 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.544671    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:06.544791 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.544738    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:06.544791 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.544769    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:06.617164 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.617130    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" event={"ID":"595853a6-5319-4c62-a6e4-a08dd2598a88","Type":"ContainerStarted","Data":"35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78"}
Jul 23 16:28:06.617306 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.617193    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" event={"ID":"595853a6-5319-4c62-a6e4-a08dd2598a88","Type":"ContainerStarted","Data":"1fd19f597dc8893f08f99648f3caefbde7f1bf4a1faa4f8910d24fc92e94228c"}
Jul 23 16:28:06.642656 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:06.642538    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" podStartSLOduration=0.642516796 podStartE2EDuration="642.516796ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:28:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:28:06.638789322 +0000 UTC m=+2349.843156717" watchObservedRunningTime="2026-07-23 16:28:06.642516796 +0000 UTC m=+2349.846884190"
Jul 23 16:28:07.393160 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:07.393126    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:07.398173 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:07.398145    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:07.622780 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:07.622755    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:07.623849 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:07.623829    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:10.392221 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.392197    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:28:10.497850 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.497782    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istiod-ca-cert\") pod \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") "
Jul 23 16:28:10.498048 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.497882    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzkdq\" (UniqueName: \"kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-kube-api-access-dzkdq\") pod \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") "
Jul 23 16:28:10.498048 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.497911    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-socket\") pod \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") "
Jul 23 16:28:10.498048 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.497975    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-credential-socket\") pod \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") "
Jul 23 16:28:10.498048 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-token\") pod \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") "
Jul 23 16:28:10.498262 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498105    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-envoy\") pod \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") "
Jul 23 16:28:10.498262 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498164    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-certs\") pod \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") "
Jul 23 16:28:10.498262 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498251    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-data\") pod \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") "
Jul 23 16:28:10.498409 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498273    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e5009bef-47da-44a5-9623-9b12cf9eb9e8" (UID: "e5009bef-47da-44a5-9623-9b12cf9eb9e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:28:10.498409 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498307    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-podinfo\") pod \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\" (UID: \"e5009bef-47da-44a5-9623-9b12cf9eb9e8\") "
Jul 23 16:28:10.498409 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498340    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e5009bef-47da-44a5-9623-9b12cf9eb9e8" (UID: "e5009bef-47da-44a5-9623-9b12cf9eb9e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:28:10.498409 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498359    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e5009bef-47da-44a5-9623-9b12cf9eb9e8" (UID: "e5009bef-47da-44a5-9623-9b12cf9eb9e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:28:10.498622 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498592    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e5009bef-47da-44a5-9623-9b12cf9eb9e8" (UID: "e5009bef-47da-44a5-9623-9b12cf9eb9e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:28:10.498715 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498673    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-credential-socket\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:28:10.498715 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498697    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istiod-ca-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:28:10.498715 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.498711    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-socket\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:28:10.499257 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.499228    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "e5009bef-47da-44a5-9623-9b12cf9eb9e8" (UID: "e5009bef-47da-44a5-9623-9b12cf9eb9e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:28:10.500536 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.500511    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "e5009bef-47da-44a5-9623-9b12cf9eb9e8" (UID: "e5009bef-47da-44a5-9623-9b12cf9eb9e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:28:10.500654 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.500535    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e5009bef-47da-44a5-9623-9b12cf9eb9e8" (UID: "e5009bef-47da-44a5-9623-9b12cf9eb9e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:28:10.500654 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.500643    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e5009bef-47da-44a5-9623-9b12cf9eb9e8" (UID: "e5009bef-47da-44a5-9623-9b12cf9eb9e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 16:28:10.500745 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.500646    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-kube-api-access-dzkdq" (OuterVolumeSpecName: "kube-api-access-dzkdq") pod "e5009bef-47da-44a5-9623-9b12cf9eb9e8" (UID: "e5009bef-47da-44a5-9623-9b12cf9eb9e8"). InnerVolumeSpecName "kube-api-access-dzkdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:28:10.599587 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.599551    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-token\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:28:10.599587 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.599580    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-envoy\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:28:10.599587 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.599589    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-workload-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:28:10.599587 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.599598    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-data\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:28:10.599901 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.599606    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5009bef-47da-44a5-9623-9b12cf9eb9e8-istio-podinfo\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:28:10.599901 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.599615    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzkdq\" (UniqueName: \"kubernetes.io/projected/e5009bef-47da-44a5-9623-9b12cf9eb9e8-kube-api-access-dzkdq\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:28:10.640644 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.640605    2562 generic.go:356] "Generic (PLEG): container finished" podID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerID="7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b" exitCode=0
Jul 23 16:28:10.640842 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.640665    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"
Jul 23 16:28:10.640842 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.640663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" event={"ID":"e5009bef-47da-44a5-9623-9b12cf9eb9e8","Type":"ContainerDied","Data":"7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b"}
Jul 23 16:28:10.640970 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.640850    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj" event={"ID":"e5009bef-47da-44a5-9623-9b12cf9eb9e8","Type":"ContainerDied","Data":"f5b92c44f3116a836ee10c6f509df07c43d9267a5265a238ed1ad0645c5a1b0f"}
Jul 23 16:28:10.640970 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.640889    2562 scope.go:117] "RemoveContainer" containerID="7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b"
Jul 23 16:28:10.650615 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.650592    2562 scope.go:117] "RemoveContainer" containerID="49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba"
Jul 23 16:28:10.660970 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.660944    2562 scope.go:117] "RemoveContainer" containerID="7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b"
Jul 23 16:28:10.661282 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:28:10.661257    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b\": container with ID starting with 7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b not found: ID does not exist" containerID="7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b"
Jul 23 16:28:10.661416 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.661287    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b"} err="failed to get container status \"7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b\": rpc error: code = NotFound desc = could not find container \"7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b\": container with ID starting with 7ea65c7a444aa7aa0739930a53877207d3f10f6021dcc784a593665b8c307e1b not found: ID does not exist"
Jul 23 16:28:10.661416 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.661306    2562 scope.go:117] "RemoveContainer" containerID="49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba"
Jul 23 16:28:10.661545 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:28:10.661522    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba\": container with ID starting with 49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba not found: ID does not exist" containerID="49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba"
Jul 23 16:28:10.661606 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.661543    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba"} err="failed to get container status \"49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba\": rpc error: code = NotFound desc = could not find container \"49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba\": container with ID starting with 49537e8d5129ef05867ce56a30e84c21f10ac30fcd11059c13d69c2706e5fbba not found: ID does not exist"
Jul 23 16:28:10.666307 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.666282    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"]
Jul 23 16:28:10.672733 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:10.672712    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-799a66-openshift-default-84f5db9d5f-4rqqj"]
Jul 23 16:28:11.254025 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:11.253988    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" path="/var/lib/kubelet/pods/e5009bef-47da-44a5-9623-9b12cf9eb9e8/volumes"
Jul 23 16:28:13.978093 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:13.978059    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"]
Jul 23 16:28:13.978534 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:13.978471    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerName="istio-proxy"
Jul 23 16:28:13.978534 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:13.978483    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerName="istio-proxy"
Jul 23 16:28:13.978680 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:13.978564    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerName="istio-proxy"
Jul 23 16:28:13.978680 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:13.978573    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerName="istio-proxy"
Jul 23 16:28:13.983229 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:13.983153    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:13.986443 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:13.986423    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-bf4b9a-openshift-default-dockercfg-rpv5s\""
Jul 23 16:28:13.997047 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:13.997021    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"]
Jul 23 16:28:14.135041 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.135000    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-data\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.135255 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.135048    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-envoy\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.135255 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.135087    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcbjw\" (UniqueName: \"kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-kube-api-access-fcbjw\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.135255 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.135114    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-token\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.135255 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.135140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-socket\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.135255 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.135166    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-credential-socket\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.135255 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.135202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-certs\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.135255 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.135225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istiod-ca-cert\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.135517 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.135263    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-podinfo\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236209 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-data\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236209 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-envoy\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236433 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236211    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcbjw\" (UniqueName: \"kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-kube-api-access-fcbjw\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236433 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236244    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-token\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236433 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236278    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-socket\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236433 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236309    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-credential-socket\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236667 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236425    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-certs\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236667 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236469    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istiod-ca-cert\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236667 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-podinfo\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236667 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236562    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-data\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236900 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-credential-socket\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.236900 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.236781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-socket\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.237236 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.237211    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istiod-ca-cert\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.237353 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.237331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-certs\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.238548 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.238532    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-envoy\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.238933 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.238915    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-podinfo\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.245776 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.245751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-token\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.245995 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.245977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcbjw\" (UniqueName: \"kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-kube-api-access-fcbjw\") pod \"e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") " pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.295847 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.295774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:14.444677 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.444645    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"]
Jul 23 16:28:14.446570 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:28:14.446544    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb1a4a5d_da43_40cd_bdc9_3b1a9c8d926e.slice/crio-a8baac04b086830a4ededc87934adc02b8a6ca6b9a5e639b7dbd41195acc6c04 WatchSource:0}: Error finding container a8baac04b086830a4ededc87934adc02b8a6ca6b9a5e639b7dbd41195acc6c04: Status 404 returned error can't find the container with id a8baac04b086830a4ededc87934adc02b8a6ca6b9a5e639b7dbd41195acc6c04
Jul 23 16:28:14.448863 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.448835    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:14.448958 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.448904    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:14.448958 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.448932    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:14.666565 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.666527    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" event={"ID":"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e","Type":"ContainerStarted","Data":"8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef"}
Jul 23 16:28:14.666766 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.666598    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" event={"ID":"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e","Type":"ContainerStarted","Data":"a8baac04b086830a4ededc87934adc02b8a6ca6b9a5e639b7dbd41195acc6c04"}
Jul 23 16:28:14.695387 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:14.695331    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" podStartSLOduration=1.695314759 podStartE2EDuration="1.695314759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:28:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:28:14.691276762 +0000 UTC m=+2357.895644156" watchObservedRunningTime="2026-07-23 16:28:14.695314759 +0000 UTC m=+2357.899682152"
Jul 23 16:28:15.296869 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:15.296834    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:15.301947 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:15.301922    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:15.671893 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:15.671865    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:15.672842 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:15.672820    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:23.665007 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:23.664958    2562 patch_prober.go:28] interesting pod/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 16:28:23.665414 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:23.665032    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 16:28:24.728196 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:24.728156    2562 generic.go:356] "Generic (PLEG): container finished" podID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerID="35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78" exitCode=0
Jul 23 16:28:24.728653 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:24.728227    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" event={"ID":"595853a6-5319-4c62-a6e4-a08dd2598a88","Type":"ContainerDied","Data":"35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78"}
Jul 23 16:28:24.728653 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:24.728650    2562 scope.go:117] "RemoveContainer" containerID="35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78"
Jul 23 16:28:24.729788 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:24.729763    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:24.729873 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:24.729848    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:24.729912 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:24.729880    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:25.746371 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:25.746330    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" event={"ID":"595853a6-5319-4c62-a6e4-a08dd2598a88","Type":"ContainerStarted","Data":"494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214"}
Jul 23 16:28:26.393455 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:26.393413    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:26.393689 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:26.393554    2562 patch_prober.go:28] interesting pod/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 23 16:28:26.393689 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:26.393606    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 23 16:28:27.393581 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:27.393535    2562 patch_prober.go:28] interesting pod/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 23 16:28:27.394120 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:27.393604    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 23 16:28:28.393756 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:28.393713    2562 patch_prober.go:28] interesting pod/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 23 16:28:28.394877 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:28.394406    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 23 16:28:29.393119 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:29.393075    2562 patch_prober.go:28] interesting pod/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 23 16:28:29.393456 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:29.393423    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 23 16:28:30.397037 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:30.396955    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:30.397505 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:30.397356    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:30.398357 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:30.398336    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:28:31.713979 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:31.713938    2562 patch_prober.go:28] interesting pod/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 16:28:31.714386 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:31.714020    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 16:28:32.797137 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:32.797058    2562 generic.go:356] "Generic (PLEG): container finished" podID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerID="8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef" exitCode=0
Jul 23 16:28:32.797528 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:32.797133    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" event={"ID":"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e","Type":"ContainerDied","Data":"8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef"}
Jul 23 16:28:32.797590 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:32.797582    2562 scope.go:117] "RemoveContainer" containerID="8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef"
Jul 23 16:28:32.798621 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:32.798589    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:32.798723 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:32.798659    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:32.798723 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:32.798691    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:28:33.810307 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:33.810270    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" event={"ID":"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e","Type":"ContainerStarted","Data":"bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212"}
Jul 23 16:28:34.296217 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:34.296179    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:34.296415 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:34.296389    2562 patch_prober.go:28] interesting pod/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 23 16:28:34.296466 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:34.296436    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 23 16:28:35.296914 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:35.296873    2562 patch_prober.go:28] interesting pod/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 23 16:28:35.297328 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:35.296933    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 23 16:28:36.296709 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:36.296666    2562 patch_prober.go:28] interesting pod/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 23 16:28:36.297006 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:36.296731    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 23 16:28:37.296668 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:37.296626    2562 patch_prober.go:28] interesting pod/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 23 16:28:37.296921 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:37.296696    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 23 16:28:38.299721 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:38.299690    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:38.300152 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:38.299994    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:38.300674 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:38.300657    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:28:56.009957 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:56.009918    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"]
Jul 23 16:28:56.010364 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:56.010177    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy" containerID="cri-o://494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214" gracePeriod=30
Jul 23 16:28:57.352149 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:57.352110    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:28:57.354486 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:57.354463    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:28:57.527424 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:28:57.527395    2562 scope.go:117] "RemoveContainer" containerID="bf019921f7335e28d328cb819916f14201288d1def5e1beb534190704ba650d1"
Jul 23 16:29:00.399612 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:00.399574    2562 patch_prober.go:28] interesting pod/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 16:29:00.400032 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:00.399637    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 16:29:01.263345 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.263317    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:29:01.352342 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.352239    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-credential-socket\") pod \"595853a6-5319-4c62-a6e4-a08dd2598a88\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") "
Jul 23 16:29:01.352342 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.352287    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-certs\") pod \"595853a6-5319-4c62-a6e4-a08dd2598a88\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") "
Jul 23 16:29:01.352342 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.352324    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/595853a6-5319-4c62-a6e4-a08dd2598a88-istiod-ca-cert\") pod \"595853a6-5319-4c62-a6e4-a08dd2598a88\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") "
Jul 23 16:29:01.352681 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.352366    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-data\") pod \"595853a6-5319-4c62-a6e4-a08dd2598a88\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") "
Jul 23 16:29:01.352681 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.352403    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-token\") pod \"595853a6-5319-4c62-a6e4-a08dd2598a88\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") "
Jul 23 16:29:01.352681 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.352457    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-podinfo\") pod \"595853a6-5319-4c62-a6e4-a08dd2598a88\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") "
Jul 23 16:29:01.352681 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.352519    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-envoy\") pod \"595853a6-5319-4c62-a6e4-a08dd2598a88\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") "
Jul 23 16:29:01.352681 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.352583    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-socket\") pod \"595853a6-5319-4c62-a6e4-a08dd2598a88\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") "
Jul 23 16:29:01.352681 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.352592    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "595853a6-5319-4c62-a6e4-a08dd2598a88" (UID: "595853a6-5319-4c62-a6e4-a08dd2598a88"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:29:01.352681 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.352654    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9f5m\" (UniqueName: \"kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-kube-api-access-v9f5m\") pod \"595853a6-5319-4c62-a6e4-a08dd2598a88\" (UID: \"595853a6-5319-4c62-a6e4-a08dd2598a88\") "
Jul 23 16:29:01.353095 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.352710    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "595853a6-5319-4c62-a6e4-a08dd2598a88" (UID: "595853a6-5319-4c62-a6e4-a08dd2598a88"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:29:01.353150 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.353115    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "595853a6-5319-4c62-a6e4-a08dd2598a88" (UID: "595853a6-5319-4c62-a6e4-a08dd2598a88"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:29:01.353322 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.353301    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-socket\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:01.353427 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.353323    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-credential-socket\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:01.353427 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.353337    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-workload-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:01.353664 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.353635    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-data" (OuterVolumeSpecName: "istio-data") pod "595853a6-5319-4c62-a6e4-a08dd2598a88" (UID: "595853a6-5319-4c62-a6e4-a08dd2598a88"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:29:01.354605 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.354533    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/595853a6-5319-4c62-a6e4-a08dd2598a88-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "595853a6-5319-4c62-a6e4-a08dd2598a88" (UID: "595853a6-5319-4c62-a6e4-a08dd2598a88"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:29:01.355558 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.355461    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "595853a6-5319-4c62-a6e4-a08dd2598a88" (UID: "595853a6-5319-4c62-a6e4-a08dd2598a88"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:29:01.355685 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.355665    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-token" (OuterVolumeSpecName: "istio-token") pod "595853a6-5319-4c62-a6e4-a08dd2598a88" (UID: "595853a6-5319-4c62-a6e4-a08dd2598a88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:29:01.355787 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.355768    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-kube-api-access-v9f5m" (OuterVolumeSpecName: "kube-api-access-v9f5m") pod "595853a6-5319-4c62-a6e4-a08dd2598a88" (UID: "595853a6-5319-4c62-a6e4-a08dd2598a88"). InnerVolumeSpecName "kube-api-access-v9f5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:29:01.356128 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.356105    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "595853a6-5319-4c62-a6e4-a08dd2598a88" (UID: "595853a6-5319-4c62-a6e4-a08dd2598a88"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 16:29:01.454178 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.454136    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9f5m\" (UniqueName: \"kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-kube-api-access-v9f5m\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:01.454178 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.454172    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/595853a6-5319-4c62-a6e4-a08dd2598a88-istiod-ca-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:01.454178 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.454186    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-data\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:01.454694 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.454197    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-token\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:01.454694 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.454209    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-podinfo\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:01.454694 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.454220    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/595853a6-5319-4c62-a6e4-a08dd2598a88-istio-envoy\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:01.967397 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.967363    2562 generic.go:356] "Generic (PLEG): container finished" podID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerID="494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214" exitCode=0
Jul 23 16:29:01.967599 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.967411    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"
Jul 23 16:29:01.967599 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.967443    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" event={"ID":"595853a6-5319-4c62-a6e4-a08dd2598a88","Type":"ContainerDied","Data":"494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214"}
Jul 23 16:29:01.967599 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.967498    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc" event={"ID":"595853a6-5319-4c62-a6e4-a08dd2598a88","Type":"ContainerDied","Data":"1fd19f597dc8893f08f99648f3caefbde7f1bf4a1faa4f8910d24fc92e94228c"}
Jul 23 16:29:01.967599 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.967520    2562 scope.go:117] "RemoveContainer" containerID="494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214"
Jul 23 16:29:01.978605 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.978585    2562 scope.go:117] "RemoveContainer" containerID="35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78"
Jul 23 16:29:01.987963 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.987943    2562 scope.go:117] "RemoveContainer" containerID="494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214"
Jul 23 16:29:01.988239 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:29:01.988222    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214\": container with ID starting with 494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214 not found: ID does not exist" containerID="494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214"
Jul 23 16:29:01.988304 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.988248    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214"} err="failed to get container status \"494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214\": rpc error: code = NotFound desc = could not find container \"494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214\": container with ID starting with 494f11d781fed1579eac59b47d6476b759a818736e14c49ae1a97b69a935e214 not found: ID does not exist"
Jul 23 16:29:01.988304 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.988264    2562 scope.go:117] "RemoveContainer" containerID="35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78"
Jul 23 16:29:01.988477 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:29:01.988462    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78\": container with ID starting with 35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78 not found: ID does not exist" containerID="35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78"
Jul 23 16:29:01.988518 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.988481    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78"} err="failed to get container status \"35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78\": rpc error: code = NotFound desc = could not find container \"35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78\": container with ID starting with 35c3ee7c74bf75b8510332b422f887429ba2089e6504b3a87b0c1a26b38e0c78 not found: ID does not exist"
Jul 23 16:29:01.992023 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.992000    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"]
Jul 23 16:29:01.999902 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:01.999878    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-89bfd8-openshift-default-6556c5dcb9-48wxc"]
Jul 23 16:29:03.254775 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:03.254735    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" path="/var/lib/kubelet/pods/595853a6-5319-4c62-a6e4-a08dd2598a88/volumes"
Jul 23 16:29:23.041011 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:23.040924    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"]
Jul 23 16:29:23.041513 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:23.041235    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy" containerID="cri-o://bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212" gracePeriod=30
Jul 23 16:29:23.302661 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:23.302569    2562 patch_prober.go:28] interesting pod/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 23 16:29:23.302661 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:23.302631    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 16:29:24.014838 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.014785    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"]
Jul 23 16:29:24.015230 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.015215    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy"
Jul 23 16:29:24.015230 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.015231    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy"
Jul 23 16:29:24.015386 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.015246    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy"
Jul 23 16:29:24.015386 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.015252    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy"
Jul 23 16:29:24.015386 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.015273    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerName="istio-proxy"
Jul 23 16:29:24.015386 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.015279    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5009bef-47da-44a5-9623-9b12cf9eb9e8" containerName="istio-proxy"
Jul 23 16:29:24.015386 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.015361    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy"
Jul 23 16:29:24.015386 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.015377    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="595853a6-5319-4c62-a6e4-a08dd2598a88" containerName="istio-proxy"
Jul 23 16:29:24.019059 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.019038    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.022771 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.022752    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-759481-openshift-default-dockercfg-vkwr8\""
Jul 23 16:29:24.032878 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.032855    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"]
Jul 23 16:29:24.177026 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.176995    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-socket\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.177026 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.177039    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-credential-socket\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.177509 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.177129    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmq4d\" (UniqueName: \"kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-kube-api-access-kmq4d\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.177509 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.177183    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-certs\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.177509 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.177221    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-istio-token\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.177509 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.177241    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-data\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.177509 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.177265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a378db54-631e-4445-9bc4-c8c93401b004-istio-podinfo\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.177509 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.177364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a378db54-631e-4445-9bc4-c8c93401b004-istiod-ca-cert\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.177509 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.177403    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-envoy\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278290 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278197    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmq4d\" (UniqueName: \"kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-kube-api-access-kmq4d\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278290 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278251    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-certs\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278290 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278272    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-istio-token\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278290 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278291    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-data\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278620 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278315    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a378db54-631e-4445-9bc4-c8c93401b004-istio-podinfo\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278620 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278342    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a378db54-631e-4445-9bc4-c8c93401b004-istiod-ca-cert\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278620 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278362    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-envoy\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278620 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278413    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-socket\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278620 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278459    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-credential-socket\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278928 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278682    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-certs\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278928 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278710    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-data\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.278928 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278882    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-socket\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.279050 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.278940    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-credential-socket\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.279152 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.279130    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a378db54-631e-4445-9bc4-c8c93401b004-istiod-ca-cert\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.280836 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.280794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-envoy\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.280992 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.280975    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a378db54-631e-4445-9bc4-c8c93401b004-istio-podinfo\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.287895 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.287874    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmq4d\" (UniqueName: \"kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-kube-api-access-kmq4d\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.289591 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.289560    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-istio-token\") pod \"e2e-shared-a-759481-openshift-default-67db877cff-vvddf\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") " pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.329520 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.329488    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:24.472336 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.472305    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"]
Jul 23 16:29:24.473581 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:29:24.473551    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda378db54_631e_4445_9bc4_c8c93401b004.slice/crio-785449642e9ef95d3c617d99cc0e27c15c17c8c1624e72a5e9b55aeb3658e7cc WatchSource:0}: Error finding container 785449642e9ef95d3c617d99cc0e27c15c17c8c1624e72a5e9b55aeb3658e7cc: Status 404 returned error can't find the container with id 785449642e9ef95d3c617d99cc0e27c15c17c8c1624e72a5e9b55aeb3658e7cc
Jul 23 16:29:24.475754 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.475722    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:24.475873 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.475784    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:24.475873 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:24.475828    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:25.096134 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:25.096101    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" event={"ID":"a378db54-631e-4445-9bc4-c8c93401b004","Type":"ContainerStarted","Data":"fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5"}
Jul 23 16:29:25.096321 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:25.096155    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" event={"ID":"a378db54-631e-4445-9bc4-c8c93401b004","Type":"ContainerStarted","Data":"785449642e9ef95d3c617d99cc0e27c15c17c8c1624e72a5e9b55aeb3658e7cc"}
Jul 23 16:29:25.120280 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:25.120228    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" podStartSLOduration=2.12021275 podStartE2EDuration="2.12021275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:29:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:29:25.117860834 +0000 UTC m=+2428.322228229" watchObservedRunningTime="2026-07-23 16:29:25.12021275 +0000 UTC m=+2428.324580216"
Jul 23 16:29:25.329620 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:25.329584    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:25.334388 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:25.334365    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:26.101236 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:26.101207    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:26.102168 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:26.102147    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:28.307697 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.307632    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:29:28.419618 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.419586    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-credential-socket\") pod \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") "
Jul 23 16:29:28.419618 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.419625    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-podinfo\") pod \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") "
Jul 23 16:29:28.419883 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.419661    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-token\") pod \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") "
Jul 23 16:29:28.419883 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.419686    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-certs\") pod \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") "
Jul 23 16:29:28.419883 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.419744    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-socket\") pod \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") "
Jul 23 16:29:28.419883 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.419789    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcbjw\" (UniqueName: \"kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-kube-api-access-fcbjw\") pod \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") "
Jul 23 16:29:28.419883 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.419866    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-envoy\") pod \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") "
Jul 23 16:29:28.420144 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.419901    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istiod-ca-cert\") pod \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") "
Jul 23 16:29:28.420144 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.419932    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" (UID: "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:29:28.420144 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.419960    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-data\") pod \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\" (UID: \"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e\") "
Jul 23 16:29:28.420144 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.420059    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" (UID: "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:29:28.420144 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.420082    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" (UID: "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:29:28.420442 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.420424    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-credential-socket\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:28.420508 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.420445    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:28.420508 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.420449    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" (UID: "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:29:28.420508 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.420461    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-workload-socket\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:28.421105 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.421077    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" (UID: "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:29:28.422196 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.422166    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" (UID: "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 16:29:28.422300 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.422238    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" (UID: "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:29:28.422300 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.422249    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" (UID: "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:29:28.422521 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.422498    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-kube-api-access-fcbjw" (OuterVolumeSpecName: "kube-api-access-fcbjw") pod "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" (UID: "eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e"). InnerVolumeSpecName "kube-api-access-fcbjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:29:28.521820 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.521781    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-data\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:28.522003 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.521839    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-podinfo\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:28.522003 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.521852    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-token\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:28.522003 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.521867    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcbjw\" (UniqueName: \"kubernetes.io/projected/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-kube-api-access-fcbjw\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:28.522003 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.521879    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istio-envoy\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:28.522003 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:28.521890    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e-istiod-ca-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:29:29.118132 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.118101    2562 generic.go:356] "Generic (PLEG): container finished" podID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerID="bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212" exitCode=0
Jul 23 16:29:29.118314 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.118138    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" event={"ID":"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e","Type":"ContainerDied","Data":"bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212"}
Jul 23 16:29:29.118314 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.118158    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"
Jul 23 16:29:29.118314 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.118180    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm" event={"ID":"eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e","Type":"ContainerDied","Data":"a8baac04b086830a4ededc87934adc02b8a6ca6b9a5e639b7dbd41195acc6c04"}
Jul 23 16:29:29.118314 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.118204    2562 scope.go:117] "RemoveContainer" containerID="bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212"
Jul 23 16:29:29.128480 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.128462    2562 scope.go:117] "RemoveContainer" containerID="8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef"
Jul 23 16:29:29.136615 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.136589    2562 scope.go:117] "RemoveContainer" containerID="bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212"
Jul 23 16:29:29.136904 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:29:29.136884    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212\": container with ID starting with bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212 not found: ID does not exist" containerID="bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212"
Jul 23 16:29:29.136968 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.136914    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212"} err="failed to get container status \"bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212\": rpc error: code = NotFound desc = could not find container \"bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212\": container with ID starting with bf2d10d618f3d5f5257c873bf3865bcea554173de866a332b308b41db54b8212 not found: ID does not exist"
Jul 23 16:29:29.136968 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.136933    2562 scope.go:117] "RemoveContainer" containerID="8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef"
Jul 23 16:29:29.137196 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:29:29.137178    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef\": container with ID starting with 8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef not found: ID does not exist" containerID="8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef"
Jul 23 16:29:29.137242 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.137202    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef"} err="failed to get container status \"8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef\": rpc error: code = NotFound desc = could not find container \"8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef\": container with ID starting with 8d7bd7c0c016390391274af7f691bb3c625e2378a7ff0c956dcc1cb6d05470ef not found: ID does not exist"
Jul 23 16:29:29.141636 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.141610    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"]
Jul 23 16:29:29.146147 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.146121    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-bf4b9a-openshift-default-6b7f86965-jwjbm"]
Jul 23 16:29:29.254065 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:29.254027    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" path="/var/lib/kubelet/pods/eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e/volumes"
Jul 23 16:29:32.343691 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.343656    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"]
Jul 23 16:29:32.344177 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.344161    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy"
Jul 23 16:29:32.344222 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.344179    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy"
Jul 23 16:29:32.344222 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.344196    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy"
Jul 23 16:29:32.344222 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.344206    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy"
Jul 23 16:29:32.344318 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.344293    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy"
Jul 23 16:29:32.348773 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.348753    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.351716 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.351693    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3a6a71-openshift-default-dockercfg-qfmhj\""
Jul 23 16:29:32.355124 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.355099    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-data\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.355258 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.355163    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-envoy\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.355309 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.355245    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81799f49-0d36-40cc-ba76-e9e14f304019-istiod-ca-cert\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.355309 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.355293    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81799f49-0d36-40cc-ba76-e9e14f304019-istio-podinfo\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.355389 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.355363    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-credential-socket\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.355435 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.355415    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-certs\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.355491 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.355446    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcmgm\" (UniqueName: \"kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-kube-api-access-xcmgm\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.355668 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.355551    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-socket\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.355668 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.355614    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-istio-token\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.359677 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.359654    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"]
Jul 23 16:29:32.456161 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-envoy\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.456161 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81799f49-0d36-40cc-ba76-e9e14f304019-istiod-ca-cert\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.456430 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456184    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81799f49-0d36-40cc-ba76-e9e14f304019-istio-podinfo\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.456430 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-credential-socket\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.456430 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456276    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-certs\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.456430 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456313    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcmgm\" (UniqueName: \"kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-kube-api-access-xcmgm\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.456430 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-socket\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.456430 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-istio-token\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.456430 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456421    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-data\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.456820 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456680    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-credential-socket\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.456820 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-data\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.456991 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.456965    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-socket\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.457086 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.457068    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81799f49-0d36-40cc-ba76-e9e14f304019-istiod-ca-cert\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.457240 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.457214    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-certs\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.458533 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.458513    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-envoy\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.458993 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.458969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81799f49-0d36-40cc-ba76-e9e14f304019-istio-podinfo\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.465588 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.465566    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-istio-token\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.465704 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.465646    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcmgm\" (UniqueName: \"kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-kube-api-access-xcmgm\") pod \"e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") " pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.661005 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.660913    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:32.807153 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.807129    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"]
Jul 23 16:29:32.809073 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:29:32.809032    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81799f49_0d36_40cc_ba76_e9e14f304019.slice/crio-8173864f3a1b6f5817b3c25a49465f64c7c2825390ab64d965049f1496bc377e WatchSource:0}: Error finding container 8173864f3a1b6f5817b3c25a49465f64c7c2825390ab64d965049f1496bc377e: Status 404 returned error can't find the container with id 8173864f3a1b6f5817b3c25a49465f64c7c2825390ab64d965049f1496bc377e
Jul 23 16:29:32.811282 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.811249    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:32.811403 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.811333    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:32.811403 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:32.811379    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:33.141971 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:33.141932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" event={"ID":"81799f49-0d36-40cc-ba76-e9e14f304019","Type":"ContainerStarted","Data":"37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f"}
Jul 23 16:29:33.142132 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:33.141990    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" event={"ID":"81799f49-0d36-40cc-ba76-e9e14f304019","Type":"ContainerStarted","Data":"8173864f3a1b6f5817b3c25a49465f64c7c2825390ab64d965049f1496bc377e"}
Jul 23 16:29:33.167061 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:33.167007    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" podStartSLOduration=1.166991574 podStartE2EDuration="1.166991574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:29:33.164147116 +0000 UTC m=+2436.368514510" watchObservedRunningTime="2026-07-23 16:29:33.166991574 +0000 UTC m=+2436.371358970"
Jul 23 16:29:33.661561 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:33.661523    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:33.666691 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:33.666663    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:34.148154 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:34.148126    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:34.148985 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:34.148965    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:42.141958 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:42.141920    2562 patch_prober.go:28] interesting pod/e2e-shared-a-759481-openshift-default-67db877cff-vvddf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 16:29:42.142360 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:42.141998    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 16:29:43.210023 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:43.209988    2562 generic.go:356] "Generic (PLEG): container finished" podID="a378db54-631e-4445-9bc4-c8c93401b004" containerID="fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5" exitCode=0
Jul 23 16:29:43.210439 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:43.210061    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" event={"ID":"a378db54-631e-4445-9bc4-c8c93401b004","Type":"ContainerDied","Data":"fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5"}
Jul 23 16:29:43.210574 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:43.210556    2562 scope.go:117] "RemoveContainer" containerID="fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5"
Jul 23 16:29:43.211554 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:43.211528    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:43.211607 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:43.211595    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:43.211643 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:43.211631    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:44.217640 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:44.217605    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" event={"ID":"a378db54-631e-4445-9bc4-c8c93401b004","Type":"ContainerStarted","Data":"a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830"}
Jul 23 16:29:44.330241 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:44.330204    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:44.330472 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:44.330445    2562 patch_prober.go:28] interesting pod/e2e-shared-a-759481-openshift-default-67db877cff-vvddf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 23 16:29:44.330528 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:44.330507    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 23 16:29:45.330380 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:45.330342    2562 patch_prober.go:28] interesting pod/e2e-shared-a-759481-openshift-default-67db877cff-vvddf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 23 16:29:45.330790 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:45.330412    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 23 16:29:46.330272 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:46.330229    2562 patch_prober.go:28] interesting pod/e2e-shared-a-759481-openshift-default-67db877cff-vvddf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 23 16:29:46.330548 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:46.330307    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 23 16:29:47.330500 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:47.330459    2562 patch_prober.go:28] interesting pod/e2e-shared-a-759481-openshift-default-67db877cff-vvddf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 23 16:29:47.330761 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:47.330532    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 23 16:29:48.333947 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:48.333908    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:48.334416 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:48.334288    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:48.335152 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:48.335134    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:29:50.189962 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:50.189927    2562 patch_prober.go:28] interesting pod/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 16:29:50.190371 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:50.189987    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 16:29:51.258117 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:51.258087    2562 generic.go:356] "Generic (PLEG): container finished" podID="81799f49-0d36-40cc-ba76-e9e14f304019" containerID="37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f" exitCode=0
Jul 23 16:29:51.258663 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:51.258156    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" event={"ID":"81799f49-0d36-40cc-ba76-e9e14f304019","Type":"ContainerDied","Data":"37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f"}
Jul 23 16:29:51.258663 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:51.258599    2562 scope.go:117] "RemoveContainer" containerID="37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f"
Jul 23 16:29:51.259646 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:51.259620    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:51.259733 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:51.259698    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:51.259772 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:51.259747    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 16:29:52.269004 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:52.268962    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" event={"ID":"81799f49-0d36-40cc-ba76-e9e14f304019","Type":"ContainerStarted","Data":"55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b"}
Jul 23 16:29:52.662043 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:52.661942    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:52.662641 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:52.662602    2562 patch_prober.go:28] interesting pod/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 23 16:29:52.662785 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:52.662665    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 23 16:29:53.661974 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:53.661929    2562 patch_prober.go:28] interesting pod/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 23 16:29:53.662496 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:53.662015    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 23 16:29:54.661777 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:54.661732    2562 patch_prober.go:28] interesting pod/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 23 16:29:54.662009 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:54.661822    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 23 16:29:55.661781 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:55.661736    2562 patch_prober.go:28] interesting pod/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 23 16:29:55.662020 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:55.661831    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 23 16:29:56.665363 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:56.665329    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:56.665833 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:56.665695    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:56.666401 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:56.666379    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:29:57.553399 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:29:57.553366    2562 scope.go:117] "RemoveContainer" containerID="b3707b0bc1720d2d3e117d0aede6bac55a27fe703974f2ed1707470737e8611d"
Jul 23 16:30:30.420040 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.419998    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"]
Jul 23 16:30:30.420645 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.420625    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb1a4a5d-da43-40cd-bdc9-3b1a9c8d926e" containerName="istio-proxy"
Jul 23 16:30:30.423576 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.423562    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.428054 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.428032    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3a6a71\"/\"openshift-service-ca.crt\""
Jul 23 16:30:30.428162 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.428032    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3a6a71\"/\"auth-test-model-3a6a71-kserve-self-signed-certs\""
Jul 23 16:30:30.428162 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.428040    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3a6a71\"/\"kube-root-ca.crt\""
Jul 23 16:30:30.428162 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.428089    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3a6a71\"/\"default-dockercfg-jdkvj\""
Jul 23 16:30:30.432712 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.432684    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"]
Jul 23 16:30:30.555001 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.554964    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tmp-dir\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.555180 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.555004    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-model-cache\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.555180 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.555027    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-home\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.555180 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.555142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-dshm\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.555293 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.555184    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tls-certs\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.555293 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.555243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lhqx\" (UniqueName: \"kubernetes.io/projected/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-kube-api-access-5lhqx\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.656034 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.655996    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhqx\" (UniqueName: \"kubernetes.io/projected/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-kube-api-access-5lhqx\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.656216 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.656087    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tmp-dir\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.656216 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.656123    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-model-cache\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.656216 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.656150    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-home\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.656347 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.656216    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-dshm\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.656347 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.656264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tls-certs\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.656604 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.656575    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-model-cache\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.656758 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.656582    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tmp-dir\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.656758 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.656645    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-home\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.658620 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.658597    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-dshm\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.658817 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.658786    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tls-certs\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.672178 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.672109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lhqx\" (UniqueName: \"kubernetes.io/projected/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-kube-api-access-5lhqx\") pod \"auth-test-model-3a6a71-kserve-5c4779d647-f48vs\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") " pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.733864 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.733810    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:30.873669 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:30.873644    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"]
Jul 23 16:30:30.875305 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:30:30.875282    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb482bbdc_81a9_45d7_8b09_0f2da6e67eca.slice/crio-d0249e59b6aa45e808fc6b3d1e5bb646a3c4b07493ebfad170a5f07452fdc92d WatchSource:0}: Error finding container d0249e59b6aa45e808fc6b3d1e5bb646a3c4b07493ebfad170a5f07452fdc92d: Status 404 returned error can't find the container with id d0249e59b6aa45e808fc6b3d1e5bb646a3c4b07493ebfad170a5f07452fdc92d
Jul 23 16:30:31.483552 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:31.483510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs" event={"ID":"b482bbdc-81a9-45d7-8b09-0f2da6e67eca","Type":"ContainerStarted","Data":"00bfc1a4b2f3167938dea5c9a2e31c550e1a36cd62a66c8fedcffa0dea2d99a0"}
Jul 23 16:30:31.483987 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:31.483565    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs" event={"ID":"b482bbdc-81a9-45d7-8b09-0f2da6e67eca","Type":"ContainerStarted","Data":"d0249e59b6aa45e808fc6b3d1e5bb646a3c4b07493ebfad170a5f07452fdc92d"}
Jul 23 16:30:31.500390 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:31.500330    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs" podStartSLOduration=1.5003145359999999 podStartE2EDuration="1.500314536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:30:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:30:31.498818263 +0000 UTC m=+2494.703185650" watchObservedRunningTime="2026-07-23 16:30:31.500314536 +0000 UTC m=+2494.704681959"
Jul 23 16:30:40.734445 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:40.734411    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:40.734917 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:40.734587    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:40.746836 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:40.746794    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:30:41.550746 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:30:41.550713    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:32:51.120837 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:32:51.120748    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"]
Jul 23 16:33:06.142664 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.142597    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs" podUID="b482bbdc-81a9-45d7-8b09-0f2da6e67eca" containerName="main" containerID="cri-o://00bfc1a4b2f3167938dea5c9a2e31c550e1a36cd62a66c8fedcffa0dea2d99a0" gracePeriod=45
Jul 23 16:33:06.337731 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.337700    2562 generic.go:356] "Generic (PLEG): container finished" podID="b482bbdc-81a9-45d7-8b09-0f2da6e67eca" containerID="00bfc1a4b2f3167938dea5c9a2e31c550e1a36cd62a66c8fedcffa0dea2d99a0" exitCode=0
Jul 23 16:33:06.337926 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.337775    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs" event={"ID":"b482bbdc-81a9-45d7-8b09-0f2da6e67eca","Type":"ContainerDied","Data":"00bfc1a4b2f3167938dea5c9a2e31c550e1a36cd62a66c8fedcffa0dea2d99a0"}
Jul 23 16:33:06.404779 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.404697    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:33:06.541541 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.541507    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-home\") pod \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") "
Jul 23 16:33:06.541541 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.541544    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tls-certs\") pod \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") "
Jul 23 16:33:06.541839 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.541586    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-model-cache\") pod \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") "
Jul 23 16:33:06.541839 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.541623    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tmp-dir\") pod \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") "
Jul 23 16:33:06.541839 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.541660    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-dshm\") pod \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") "
Jul 23 16:33:06.541839 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.541718    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lhqx\" (UniqueName: \"kubernetes.io/projected/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-kube-api-access-5lhqx\") pod \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\" (UID: \"b482bbdc-81a9-45d7-8b09-0f2da6e67eca\") "
Jul 23 16:33:06.542060 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.541864    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-home" (OuterVolumeSpecName: "home") pod "b482bbdc-81a9-45d7-8b09-0f2da6e67eca" (UID: "b482bbdc-81a9-45d7-8b09-0f2da6e67eca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:33:06.542060 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.541917    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-model-cache" (OuterVolumeSpecName: "model-cache") pod "b482bbdc-81a9-45d7-8b09-0f2da6e67eca" (UID: "b482bbdc-81a9-45d7-8b09-0f2da6e67eca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:33:06.542060 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.541929    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b482bbdc-81a9-45d7-8b09-0f2da6e67eca" (UID: "b482bbdc-81a9-45d7-8b09-0f2da6e67eca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:33:06.542169 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.542119    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-home\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:33:06.542169 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.542131    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-model-cache\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:33:06.542169 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.542141    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tmp-dir\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:33:06.544073 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.544039    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b482bbdc-81a9-45d7-8b09-0f2da6e67eca" (UID: "b482bbdc-81a9-45d7-8b09-0f2da6e67eca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:33:06.544160 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.544132    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-kube-api-access-5lhqx" (OuterVolumeSpecName: "kube-api-access-5lhqx") pod "b482bbdc-81a9-45d7-8b09-0f2da6e67eca" (UID: "b482bbdc-81a9-45d7-8b09-0f2da6e67eca"). InnerVolumeSpecName "kube-api-access-5lhqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:33:06.544201 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.544157    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-dshm" (OuterVolumeSpecName: "dshm") pod "b482bbdc-81a9-45d7-8b09-0f2da6e67eca" (UID: "b482bbdc-81a9-45d7-8b09-0f2da6e67eca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:33:06.642716 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.642679    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:33:06.642716 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.642709    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-dshm\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:33:06.642716 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:06.642719    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lhqx\" (UniqueName: \"kubernetes.io/projected/b482bbdc-81a9-45d7-8b09-0f2da6e67eca-kube-api-access-5lhqx\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:33:07.344519 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:07.344490    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"
Jul 23 16:33:07.344960 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:07.344490    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs" event={"ID":"b482bbdc-81a9-45d7-8b09-0f2da6e67eca","Type":"ContainerDied","Data":"d0249e59b6aa45e808fc6b3d1e5bb646a3c4b07493ebfad170a5f07452fdc92d"}
Jul 23 16:33:07.344960 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:07.344613    2562 scope.go:117] "RemoveContainer" containerID="00bfc1a4b2f3167938dea5c9a2e31c550e1a36cd62a66c8fedcffa0dea2d99a0"
Jul 23 16:33:07.362687 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:07.362653    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"]
Jul 23 16:33:07.366633 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:07.366608    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3a6a71/auth-test-model-3a6a71-kserve-5c4779d647-f48vs"]
Jul 23 16:33:09.255255 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:09.255216    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b482bbdc-81a9-45d7-8b09-0f2da6e67eca" path="/var/lib/kubelet/pods/b482bbdc-81a9-45d7-8b09-0f2da6e67eca/volumes"
Jul 23 16:33:57.395007 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:57.394923    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:33:57.396062 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:57.396044    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:33:57.609908 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:33:57.609875    2562 scope.go:117] "RemoveContainer" containerID="09b99d28b950d16d648a295a0dec497e1b0ce3b326ff77138d5c0fe48093b738"
Jul 23 16:34:17.594044 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.594002    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"]
Jul 23 16:34:17.594604 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.594562    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b482bbdc-81a9-45d7-8b09-0f2da6e67eca" containerName="main"
Jul 23 16:34:17.594604 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.594579    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b482bbdc-81a9-45d7-8b09-0f2da6e67eca" containerName="main"
Jul 23 16:34:17.594683 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.594652    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b482bbdc-81a9-45d7-8b09-0f2da6e67eca" containerName="main"
Jul 23 16:34:17.597930 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.597912    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.600564 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.600540    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-759481\"/\"openshift-service-ca.crt\""
Jul 23 16:34:17.601790 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.601770    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-759481\"/\"default-dockercfg-s7m5b\""
Jul 23 16:34:17.601899 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.601773    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-759481\"/\"kube-root-ca.crt\""
Jul 23 16:34:17.601899 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.601883    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-759481\"/\"rate-test-model-759481-kserve-self-signed-certs\""
Jul 23 16:34:17.607503 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.607484    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"]
Jul 23 16:34:17.647785 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.647748    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-model-cache\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.647969 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.647812    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48mph\" (UniqueName: \"kubernetes.io/projected/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-kube-api-access-48mph\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.647969 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.647855    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-dshm\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.647969 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.647949    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tls-certs\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.648085 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.647986    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tmp-dir\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.648085 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.648017    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-home\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.748727 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.748688    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tls-certs\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.748931 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.748737    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tmp-dir\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.748931 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.748863    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-home\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.748931 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.748927    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-model-cache\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.749106 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.748968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48mph\" (UniqueName: \"kubernetes.io/projected/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-kube-api-access-48mph\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.749106 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.748994    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-dshm\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.749212 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.749158    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tmp-dir\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.749212 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.749179    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-home\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.749305 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.749248    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-model-cache\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.751070 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.751048    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-dshm\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.751468 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.751449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tls-certs\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.757850 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.757827    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48mph\" (UniqueName: \"kubernetes.io/projected/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-kube-api-access-48mph\") pod \"rate-test-model-759481-kserve-5fd974fddc-cn8hw\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") " pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:17.908143 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:17.908053    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:18.052874 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:18.052847    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"]
Jul 23 16:34:18.055210 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:34:18.055174    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f5e1e64_6a45_4a4f_8dac_7ad8a4d05769.slice/crio-a4da51a129e2ae51d64d4c488b839244b879ccc1a521d1a97bcd8c5b1d9a3e3a WatchSource:0}: Error finding container a4da51a129e2ae51d64d4c488b839244b879ccc1a521d1a97bcd8c5b1d9a3e3a: Status 404 returned error can't find the container with id a4da51a129e2ae51d64d4c488b839244b879ccc1a521d1a97bcd8c5b1d9a3e3a
Jul 23 16:34:18.731333 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:18.731301    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw" event={"ID":"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769","Type":"ContainerStarted","Data":"a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6"}
Jul 23 16:34:18.731333 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:18.731341    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw" event={"ID":"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769","Type":"ContainerStarted","Data":"a4da51a129e2ae51d64d4c488b839244b879ccc1a521d1a97bcd8c5b1d9a3e3a"}
Jul 23 16:34:18.750156 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:18.750101    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw" podStartSLOduration=1.7500847510000002 podStartE2EDuration="1.750084751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:34:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:34:18.748459393 +0000 UTC m=+2721.952826788" watchObservedRunningTime="2026-07-23 16:34:18.750084751 +0000 UTC m=+2721.954452165"
Jul 23 16:34:27.908470 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:27.908410    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:27.909036 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:27.908496    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:27.920981 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:27.920957    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:34:28.794625 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:34:28.794595    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:35:32.084131 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:32.083937    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"]
Jul 23 16:35:47.105634 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.105571    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw" podUID="9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" containerName="main" containerID="cri-o://a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6" gracePeriod=45
Jul 23 16:35:47.356473 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.356406    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:35:47.549595 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.549562    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tls-certs\") pod \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") "
Jul 23 16:35:47.549783 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.549610    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-model-cache\") pod \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") "
Jul 23 16:35:47.549783 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.549631    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48mph\" (UniqueName: \"kubernetes.io/projected/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-kube-api-access-48mph\") pod \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") "
Jul 23 16:35:47.549783 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.549761    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-home\") pod \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") "
Jul 23 16:35:47.550001 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.549838    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-dshm\") pod \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") "
Jul 23 16:35:47.550001 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.549879    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tmp-dir\") pod \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\" (UID: \"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769\") "
Jul 23 16:35:47.550001 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.549908    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" (UID: "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:35:47.550161 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.550042    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-home" (OuterVolumeSpecName: "home") pod "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" (UID: "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:35:47.550216 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.550180    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" (UID: "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:35:47.550512 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.550496    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-model-cache\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:35:47.550568 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.550517    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-home\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:35:47.550568 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.550528    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tmp-dir\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:35:47.552014 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.551987    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-dshm" (OuterVolumeSpecName: "dshm") pod "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" (UID: "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:35:47.552166 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.552145    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" (UID: "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 16:35:47.552236 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.552168    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-kube-api-access-48mph" (OuterVolumeSpecName: "kube-api-access-48mph") pod "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" (UID: "9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769"). InnerVolumeSpecName "kube-api-access-48mph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:35:47.651868 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.651777    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:35:47.651868 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.651831    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48mph\" (UniqueName: \"kubernetes.io/projected/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-kube-api-access-48mph\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:35:47.651868 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:47.651841    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769-dshm\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:35:48.213608 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:48.213575    2562 generic.go:356] "Generic (PLEG): container finished" podID="9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" containerID="a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6" exitCode=0
Jul 23 16:35:48.214062 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:48.213624    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"
Jul 23 16:35:48.214062 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:48.213659    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw" event={"ID":"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769","Type":"ContainerDied","Data":"a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6"}
Jul 23 16:35:48.214062 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:48.213705    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw" event={"ID":"9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769","Type":"ContainerDied","Data":"a4da51a129e2ae51d64d4c488b839244b879ccc1a521d1a97bcd8c5b1d9a3e3a"}
Jul 23 16:35:48.214062 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:48.213729    2562 scope.go:117] "RemoveContainer" containerID="a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6"
Jul 23 16:35:48.223428 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:48.223405    2562 scope.go:117] "RemoveContainer" containerID="a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6"
Jul 23 16:35:48.223687 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:35:48.223668    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6\": container with ID starting with a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6 not found: ID does not exist" containerID="a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6"
Jul 23 16:35:48.223727 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:48.223697    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6"} err="failed to get container status \"a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6\": rpc error: code = NotFound desc = could not find container \"a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6\": container with ID starting with a5f28331e72a573b5843dbcda14a1189d21232348e2be26b2a3ed8995ea2d6e6 not found: ID does not exist"
Jul 23 16:35:48.236550 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:48.236510    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"]
Jul 23 16:35:48.239813 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:48.239773    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-759481/rate-test-model-759481-kserve-5fd974fddc-cn8hw"]
Jul 23 16:35:49.255229 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:35:49.255193    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" path="/var/lib/kubelet/pods/9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769/volumes"
Jul 23 16:36:14.551622 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:14.551587    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"]
Jul 23 16:36:14.552285 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:14.551780    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy" containerID="cri-o://a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830" gracePeriod=30
Jul 23 16:36:18.336979 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:18.336941    2562 patch_prober.go:28] interesting pod/e2e-shared-a-759481-openshift-default-67db877cff-vvddf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 16:36:18.337386 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:18.337002    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 16:36:19.809048 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.809020    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:36:19.966988 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.966947    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-certs\") pod \"a378db54-631e-4445-9bc4-c8c93401b004\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") "
Jul 23 16:36:19.966988 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.966996    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-credential-socket\") pod \"a378db54-631e-4445-9bc4-c8c93401b004\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") "
Jul 23 16:36:19.967264 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967032    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-data\") pod \"a378db54-631e-4445-9bc4-c8c93401b004\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") "
Jul 23 16:36:19.967264 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967061    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-istio-token\") pod \"a378db54-631e-4445-9bc4-c8c93401b004\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") "
Jul 23 16:36:19.967264 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967083    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-envoy\") pod \"a378db54-631e-4445-9bc4-c8c93401b004\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") "
Jul 23 16:36:19.967264 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967135    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmq4d\" (UniqueName: \"kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-kube-api-access-kmq4d\") pod \"a378db54-631e-4445-9bc4-c8c93401b004\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") "
Jul 23 16:36:19.967264 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967214    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a378db54-631e-4445-9bc4-c8c93401b004-istiod-ca-cert\") pod \"a378db54-631e-4445-9bc4-c8c93401b004\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") "
Jul 23 16:36:19.967264 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967248    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-socket\") pod \"a378db54-631e-4445-9bc4-c8c93401b004\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") "
Jul 23 16:36:19.967519 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967296    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a378db54-631e-4445-9bc4-c8c93401b004" (UID: "a378db54-631e-4445-9bc4-c8c93401b004"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:36:19.967519 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967315    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a378db54-631e-4445-9bc4-c8c93401b004" (UID: "a378db54-631e-4445-9bc4-c8c93401b004"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:36:19.967519 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967355    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a378db54-631e-4445-9bc4-c8c93401b004-istio-podinfo\") pod \"a378db54-631e-4445-9bc4-c8c93401b004\" (UID: \"a378db54-631e-4445-9bc4-c8c93401b004\") "
Jul 23 16:36:19.967519 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967462    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a378db54-631e-4445-9bc4-c8c93401b004" (UID: "a378db54-631e-4445-9bc4-c8c93401b004"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:36:19.967766 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967745    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:19.967858 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967770    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-credential-socket\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:19.967858 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967783    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-workload-socket\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:19.967973 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.967939    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a378db54-631e-4445-9bc4-c8c93401b004-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a378db54-631e-4445-9bc4-c8c93401b004" (UID: "a378db54-631e-4445-9bc4-c8c93401b004"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:36:19.968178 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.968140    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-data" (OuterVolumeSpecName: "istio-data") pod "a378db54-631e-4445-9bc4-c8c93401b004" (UID: "a378db54-631e-4445-9bc4-c8c93401b004"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:36:19.969524 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.969501    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a378db54-631e-4445-9bc4-c8c93401b004" (UID: "a378db54-631e-4445-9bc4-c8c93401b004"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:36:19.969636 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.969566    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-kube-api-access-kmq4d" (OuterVolumeSpecName: "kube-api-access-kmq4d") pod "a378db54-631e-4445-9bc4-c8c93401b004" (UID: "a378db54-631e-4445-9bc4-c8c93401b004"). InnerVolumeSpecName "kube-api-access-kmq4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:36:19.969707 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.969689    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-istio-token" (OuterVolumeSpecName: "istio-token") pod "a378db54-631e-4445-9bc4-c8c93401b004" (UID: "a378db54-631e-4445-9bc4-c8c93401b004"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:36:19.969752 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:19.969713    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a378db54-631e-4445-9bc4-c8c93401b004-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a378db54-631e-4445-9bc4-c8c93401b004" (UID: "a378db54-631e-4445-9bc4-c8c93401b004"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 16:36:20.069241 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.069195    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-data\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:20.069241 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.069235    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-istio-token\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:20.069241 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.069243    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a378db54-631e-4445-9bc4-c8c93401b004-istio-envoy\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:20.069241 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.069252    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmq4d\" (UniqueName: \"kubernetes.io/projected/a378db54-631e-4445-9bc4-c8c93401b004-kube-api-access-kmq4d\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:20.069612 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.069263    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a378db54-631e-4445-9bc4-c8c93401b004-istiod-ca-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:20.069612 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.069272    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a378db54-631e-4445-9bc4-c8c93401b004-istio-podinfo\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:20.382317 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.382233    2562 generic.go:356] "Generic (PLEG): container finished" podID="a378db54-631e-4445-9bc4-c8c93401b004" containerID="a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830" exitCode=0
Jul 23 16:36:20.382317 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.382289    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"
Jul 23 16:36:20.382501 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.382319    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" event={"ID":"a378db54-631e-4445-9bc4-c8c93401b004","Type":"ContainerDied","Data":"a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830"}
Jul 23 16:36:20.382501 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.382383    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf" event={"ID":"a378db54-631e-4445-9bc4-c8c93401b004","Type":"ContainerDied","Data":"785449642e9ef95d3c617d99cc0e27c15c17c8c1624e72a5e9b55aeb3658e7cc"}
Jul 23 16:36:20.382501 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.382415    2562 scope.go:117] "RemoveContainer" containerID="a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830"
Jul 23 16:36:20.392320 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.392303    2562 scope.go:117] "RemoveContainer" containerID="fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5"
Jul 23 16:36:20.400463 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.400444    2562 scope.go:117] "RemoveContainer" containerID="a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830"
Jul 23 16:36:20.400759 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:36:20.400737    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830\": container with ID starting with a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830 not found: ID does not exist" containerID="a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830"
Jul 23 16:36:20.400882 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.400766    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830"} err="failed to get container status \"a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830\": rpc error: code = NotFound desc = could not find container \"a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830\": container with ID starting with a112cb2bdb93ee8e678a1f9b60867d2c133360959c767dcb2df2e69acea67830 not found: ID does not exist"
Jul 23 16:36:20.400882 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.400785    2562 scope.go:117] "RemoveContainer" containerID="fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5"
Jul 23 16:36:20.401054 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:36:20.401033    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5\": container with ID starting with fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5 not found: ID does not exist" containerID="fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5"
Jul 23 16:36:20.401093 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.401063    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5"} err="failed to get container status \"fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5\": rpc error: code = NotFound desc = could not find container \"fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5\": container with ID starting with fb4803d670120874fcd2461f88b77fd6c6f1fafdbb2f4aef9f7dc722ad3127e5 not found: ID does not exist"
Jul 23 16:36:20.406265 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.406241    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"]
Jul 23 16:36:20.408166 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:20.408147    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-759481-openshift-default-67db877cff-vvddf"]
Jul 23 16:36:21.254166 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:21.254134    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a378db54-631e-4445-9bc4-c8c93401b004" path="/var/lib/kubelet/pods/a378db54-631e-4445-9bc4-c8c93401b004/volumes"
Jul 23 16:36:36.500623 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:36.500587    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"]
Jul 23 16:36:36.501143 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:36.500867    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy" containerID="cri-o://55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b" gracePeriod=30
Jul 23 16:36:41.756501 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.756472    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:36:41.869986 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.869897    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81799f49-0d36-40cc-ba76-e9e14f304019-istiod-ca-cert\") pod \"81799f49-0d36-40cc-ba76-e9e14f304019\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") "
Jul 23 16:36:41.869986 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.869936    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcmgm\" (UniqueName: \"kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-kube-api-access-xcmgm\") pod \"81799f49-0d36-40cc-ba76-e9e14f304019\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") "
Jul 23 16:36:41.869986 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.869980    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-certs\") pod \"81799f49-0d36-40cc-ba76-e9e14f304019\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") "
Jul 23 16:36:41.870286 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870036    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-envoy\") pod \"81799f49-0d36-40cc-ba76-e9e14f304019\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") "
Jul 23 16:36:41.870286 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870065    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-socket\") pod \"81799f49-0d36-40cc-ba76-e9e14f304019\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") "
Jul 23 16:36:41.870286 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870150    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-data\") pod \"81799f49-0d36-40cc-ba76-e9e14f304019\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") "
Jul 23 16:36:41.870286 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870193    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81799f49-0d36-40cc-ba76-e9e14f304019-istio-podinfo\") pod \"81799f49-0d36-40cc-ba76-e9e14f304019\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") "
Jul 23 16:36:41.870286 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870219    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-istio-token\") pod \"81799f49-0d36-40cc-ba76-e9e14f304019\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") "
Jul 23 16:36:41.870286 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870245    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-credential-socket\") pod \"81799f49-0d36-40cc-ba76-e9e14f304019\" (UID: \"81799f49-0d36-40cc-ba76-e9e14f304019\") "
Jul 23 16:36:41.870602 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870369    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "81799f49-0d36-40cc-ba76-e9e14f304019" (UID: "81799f49-0d36-40cc-ba76-e9e14f304019"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:36:41.870602 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870433    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81799f49-0d36-40cc-ba76-e9e14f304019-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "81799f49-0d36-40cc-ba76-e9e14f304019" (UID: "81799f49-0d36-40cc-ba76-e9e14f304019"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 16:36:41.870602 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870435    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "81799f49-0d36-40cc-ba76-e9e14f304019" (UID: "81799f49-0d36-40cc-ba76-e9e14f304019"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:36:41.870778 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870629    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81799f49-0d36-40cc-ba76-e9e14f304019-istiod-ca-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:41.870778 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870649    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:41.870778 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870688    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-workload-socket\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:41.870778 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870766    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "81799f49-0d36-40cc-ba76-e9e14f304019" (UID: "81799f49-0d36-40cc-ba76-e9e14f304019"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:36:41.871044 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.870956    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-data" (OuterVolumeSpecName: "istio-data") pod "81799f49-0d36-40cc-ba76-e9e14f304019" (UID: "81799f49-0d36-40cc-ba76-e9e14f304019"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:36:41.872496 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.872471    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81799f49-0d36-40cc-ba76-e9e14f304019-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "81799f49-0d36-40cc-ba76-e9e14f304019" (UID: "81799f49-0d36-40cc-ba76-e9e14f304019"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 16:36:41.872496 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.872470    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "81799f49-0d36-40cc-ba76-e9e14f304019" (UID: "81799f49-0d36-40cc-ba76-e9e14f304019"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 16:36:41.872651 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.872499    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-kube-api-access-xcmgm" (OuterVolumeSpecName: "kube-api-access-xcmgm") pod "81799f49-0d36-40cc-ba76-e9e14f304019" (UID: "81799f49-0d36-40cc-ba76-e9e14f304019"). InnerVolumeSpecName "kube-api-access-xcmgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:36:41.872651 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.872533    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-istio-token" (OuterVolumeSpecName: "istio-token") pod "81799f49-0d36-40cc-ba76-e9e14f304019" (UID: "81799f49-0d36-40cc-ba76-e9e14f304019"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 16:36:41.971963 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.971925    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-envoy\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:41.971963 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.971955    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-istio-data\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:41.971963 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.971968    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81799f49-0d36-40cc-ba76-e9e14f304019-istio-podinfo\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:41.972215 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.971980    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-istio-token\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:41.972215 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.971992    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81799f49-0d36-40cc-ba76-e9e14f304019-credential-socket\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:41.972215 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:41.972003    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcmgm\" (UniqueName: \"kubernetes.io/projected/81799f49-0d36-40cc-ba76-e9e14f304019-kube-api-access-xcmgm\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Jul 23 16:36:42.501552 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.501516    2562 generic.go:356] "Generic (PLEG): container finished" podID="81799f49-0d36-40cc-ba76-e9e14f304019" containerID="55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b" exitCode=0
Jul 23 16:36:42.501751 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.501572    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"
Jul 23 16:36:42.501751 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.501605    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" event={"ID":"81799f49-0d36-40cc-ba76-e9e14f304019","Type":"ContainerDied","Data":"55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b"}
Jul 23 16:36:42.501751 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.501669    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" event={"ID":"81799f49-0d36-40cc-ba76-e9e14f304019","Type":"ContainerDied","Data":"8173864f3a1b6f5817b3c25a49465f64c7c2825390ab64d965049f1496bc377e"}
Jul 23 16:36:42.501751 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.501700    2562 scope.go:117] "RemoveContainer" containerID="55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b"
Jul 23 16:36:42.513738 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.513713    2562 scope.go:117] "RemoveContainer" containerID="37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f"
Jul 23 16:36:42.522086 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.522065    2562 scope.go:117] "RemoveContainer" containerID="55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b"
Jul 23 16:36:42.522362 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:36:42.522341    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b\": container with ID starting with 55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b not found: ID does not exist" containerID="55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b"
Jul 23 16:36:42.522439 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.522372    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b"} err="failed to get container status \"55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b\": rpc error: code = NotFound desc = could not find container \"55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b\": container with ID starting with 55afb336126fa02cd14cb3f3d0fd945dde80dfc72762c41e27487b563a37836b not found: ID does not exist"
Jul 23 16:36:42.522439 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.522390    2562 scope.go:117] "RemoveContainer" containerID="37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f"
Jul 23 16:36:42.522622 ip-10-0-140-230 kubenswrapper[2562]: E0723 16:36:42.522600    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f\": container with ID starting with 37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f not found: ID does not exist" containerID="37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f"
Jul 23 16:36:42.522664 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.522633    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f"} err="failed to get container status \"37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f\": rpc error: code = NotFound desc = could not find container \"37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f\": container with ID starting with 37b3b757cdc40e4238c08627df176782c42d89286f7849a0864e9b180ff0964f not found: ID does not exist"
Jul 23 16:36:42.526832 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.526794    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"]
Jul 23 16:36:42.534042 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.534016    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn"]
Jul 23 16:36:42.666899 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.666859    2562 patch_prober.go:28] interesting pod/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 16:36:42.667085 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:42.666948    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3a6a71-openshift-default-5cd74ff4c9-wxshn" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 16:36:43.254670 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:43.254635    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" path="/var/lib/kubelet/pods/81799f49-0d36-40cc-ba76-e9e14f304019/volumes"
Jul 23 16:36:43.263928 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:43.263903    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8498cdcf8c-dws5b_0dbaf1a1-09be-4db8-92fe-d1a17ed4437e/manager/0.log"
Jul 23 16:36:43.961395 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:43.961366    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-646b5d5c79-tdxr9_d871218f-5819-44e3-9009-d2a196e35031/copy-manifests/0.log"
Jul 23 16:36:43.974538 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:43.974506    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-646b5d5c79-tdxr9_d871218f-5819-44e3-9009-d2a196e35031/manager/0.log"
Jul 23 16:36:44.487296 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:44.487271    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747040-mt6w6_7d416850-9072-4567-94ea-a795495a810b/cleanup/0.log"
Jul 23 16:36:44.619080 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:44.619049    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747055-9lfv6_7ab27902-ab28-4ce4-adb9-1ab7cb149f22/cleanup/0.log"
Jul 23 16:36:46.570076 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:46.570048    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-r8k4p_f4617220-9992-4496-98e1-b0c489035f1b/manager/0.log"
Jul 23 16:36:46.694578 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:46.694551    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mbsjf_ec736c46-6305-4a0d-ae9a-5b83ac485177/kuadrant-console-plugin/0.log"
Jul 23 16:36:46.952503 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:46.952469    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-p799l_393a52f5-014c-4c5c-b1e7-654aa169a5da/manager/0.log"
Jul 23 16:36:47.859178 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:47.859150    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fb5dcd56-rwpjm_7158c456-141f-4e29-8ce3-711b500e2b1a/kube-auth-proxy/0.log"
Jul 23 16:36:48.888082 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:48.887987    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-jqnrc_4e416ac9-c394-4517-b5b9-359ef1f4e8cf/main/0.log"
Jul 23 16:36:55.831026 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.830993    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4nhtn/must-gather-hj6jm"]
Jul 23 16:36:55.831544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831419    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy"
Jul 23 16:36:55.831544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831431    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy"
Jul 23 16:36:55.831544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831442    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy"
Jul 23 16:36:55.831544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831449    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy"
Jul 23 16:36:55.831544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831456    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy"
Jul 23 16:36:55.831544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831462    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy"
Jul 23 16:36:55.831544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831468    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" containerName="main"
Jul 23 16:36:55.831544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831473    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" containerName="main"
Jul 23 16:36:55.831544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831488    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy"
Jul 23 16:36:55.831544 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831494    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy"
Jul 23 16:36:55.831951 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831552    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f5e1e64-6a45-4a4f-8dac-7ad8a4d05769" containerName="main"
Jul 23 16:36:55.831951 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831564    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy"
Jul 23 16:36:55.831951 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831571    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy"
Jul 23 16:36:55.831951 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831577    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a378db54-631e-4445-9bc4-c8c93401b004" containerName="istio-proxy"
Jul 23 16:36:55.831951 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.831582    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="81799f49-0d36-40cc-ba76-e9e14f304019" containerName="istio-proxy"
Jul 23 16:36:55.836062 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.836046    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nhtn/must-gather-hj6jm"
Jul 23 16:36:55.839315 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.839293    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4nhtn\"/\"kube-root-ca.crt\""
Jul 23 16:36:55.839315 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.839308    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4nhtn\"/\"default-dockercfg-8r4ch\""
Jul 23 16:36:55.840474 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.840446    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4nhtn\"/\"openshift-service-ca.crt\""
Jul 23 16:36:55.848310 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.848278    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nhtn/must-gather-hj6jm"]
Jul 23 16:36:55.905857 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.905793    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5db66\" (UniqueName: \"kubernetes.io/projected/391d1584-8722-462d-a195-489cd2a5ca1f-kube-api-access-5db66\") pod \"must-gather-hj6jm\" (UID: \"391d1584-8722-462d-a195-489cd2a5ca1f\") " pod="openshift-must-gather-4nhtn/must-gather-hj6jm"
Jul 23 16:36:55.905857 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:55.905862    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/391d1584-8722-462d-a195-489cd2a5ca1f-must-gather-output\") pod \"must-gather-hj6jm\" (UID: \"391d1584-8722-462d-a195-489cd2a5ca1f\") " pod="openshift-must-gather-4nhtn/must-gather-hj6jm"
Jul 23 16:36:56.007030 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:56.006989    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5db66\" (UniqueName: \"kubernetes.io/projected/391d1584-8722-462d-a195-489cd2a5ca1f-kube-api-access-5db66\") pod \"must-gather-hj6jm\" (UID: \"391d1584-8722-462d-a195-489cd2a5ca1f\") " pod="openshift-must-gather-4nhtn/must-gather-hj6jm"
Jul 23 16:36:56.007030 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:56.007034    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/391d1584-8722-462d-a195-489cd2a5ca1f-must-gather-output\") pod \"must-gather-hj6jm\" (UID: \"391d1584-8722-462d-a195-489cd2a5ca1f\") " pod="openshift-must-gather-4nhtn/must-gather-hj6jm"
Jul 23 16:36:56.007447 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:56.007425    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/391d1584-8722-462d-a195-489cd2a5ca1f-must-gather-output\") pod \"must-gather-hj6jm\" (UID: \"391d1584-8722-462d-a195-489cd2a5ca1f\") " pod="openshift-must-gather-4nhtn/must-gather-hj6jm"
Jul 23 16:36:56.015563 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:56.015528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5db66\" (UniqueName: \"kubernetes.io/projected/391d1584-8722-462d-a195-489cd2a5ca1f-kube-api-access-5db66\") pod \"must-gather-hj6jm\" (UID: \"391d1584-8722-462d-a195-489cd2a5ca1f\") " pod="openshift-must-gather-4nhtn/must-gather-hj6jm"
Jul 23 16:36:56.153423 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:56.153328    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nhtn/must-gather-hj6jm"
Jul 23 16:36:56.288197 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:56.288167    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nhtn/must-gather-hj6jm"]
Jul 23 16:36:56.290153 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:36:56.290120    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod391d1584_8722_462d_a195_489cd2a5ca1f.slice/crio-30dfe8400bdadd8d5fbd3833de3e6349ec6ee65558ae302a1f748ab14b46a5af WatchSource:0}: Error finding container 30dfe8400bdadd8d5fbd3833de3e6349ec6ee65558ae302a1f748ab14b46a5af: Status 404 returned error can't find the container with id 30dfe8400bdadd8d5fbd3833de3e6349ec6ee65558ae302a1f748ab14b46a5af
Jul 23 16:36:56.292095 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:56.292072    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 16:36:56.572611 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:56.572576    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nhtn/must-gather-hj6jm" event={"ID":"391d1584-8722-462d-a195-489cd2a5ca1f","Type":"ContainerStarted","Data":"30dfe8400bdadd8d5fbd3833de3e6349ec6ee65558ae302a1f748ab14b46a5af"}
Jul 23 16:36:57.582519 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:57.582407    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nhtn/must-gather-hj6jm" event={"ID":"391d1584-8722-462d-a195-489cd2a5ca1f","Type":"ContainerStarted","Data":"d7b1c8fc24989024fc82cc68af408e205704f8e1086bfb9789fd0e43bb891707"}
Jul 23 16:36:57.582519 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:57.582469    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nhtn/must-gather-hj6jm" event={"ID":"391d1584-8722-462d-a195-489cd2a5ca1f","Type":"ContainerStarted","Data":"4e0582ea42b15f85cf06e331e6b6f5861daf68a50f9a1b3da126658f6c0798b1"}
Jul 23 16:36:57.599608 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:57.599555    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4nhtn/must-gather-hj6jm" podStartSLOduration=1.7037453070000002 podStartE2EDuration="2.599537933s" totalImagesPullingTime="895.792626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:36:57.59763458 +0000 UTC m=+2880.802001968" watchObservedRunningTime="2026-07-23 16:36:57.599537933 +0000 UTC m=+2880.803905356"
Jul 23 16:36:58.868231 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:58.868203    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sl96g_9b20f8df-5ab7-4534-8ae3-f51228f128b8/global-pull-secret-syncer/0.log"
Jul 23 16:36:58.953496 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:58.953465    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-99t5l_d451b749-46af-4de3-9409-ad4978d400ee/konnectivity-agent/0.log"
Jul 23 16:36:59.074680 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:36:59.074650    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-230.ec2.internal_ce049313ba5611f719bb6a5b420aca48/haproxy/0.log"
Jul 23 16:37:04.641488 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:04.641396    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-r8k4p_f4617220-9992-4496-98e1-b0c489035f1b/manager/0.log"
Jul 23 16:37:04.672880 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:04.672759    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mbsjf_ec736c46-6305-4a0d-ae9a-5b83ac485177/kuadrant-console-plugin/0.log"
Jul 23 16:37:04.938794 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:04.938691    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-p799l_393a52f5-014c-4c5c-b1e7-654aa169a5da/manager/0.log"
Jul 23 16:37:06.677167 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:06.677131    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_05d673e3-59ac-4120-8a1a-90bd25c63823/alertmanager/0.log"
Jul 23 16:37:06.701539 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:06.701511    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_05d673e3-59ac-4120-8a1a-90bd25c63823/config-reloader/0.log"
Jul 23 16:37:06.735409 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:06.735381    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_05d673e3-59ac-4120-8a1a-90bd25c63823/kube-rbac-proxy-web/0.log"
Jul 23 16:37:06.764568 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:06.764454    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_05d673e3-59ac-4120-8a1a-90bd25c63823/kube-rbac-proxy/0.log"
Jul 23 16:37:06.786410 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:06.786372    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_05d673e3-59ac-4120-8a1a-90bd25c63823/kube-rbac-proxy-metric/0.log"
Jul 23 16:37:06.809768 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:06.809740    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_05d673e3-59ac-4120-8a1a-90bd25c63823/prom-label-proxy/0.log"
Jul 23 16:37:06.832061 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:06.832034    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_05d673e3-59ac-4120-8a1a-90bd25c63823/init-config-reloader/0.log"
Jul 23 16:37:06.917823 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:06.917104    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-6b857_e46ca500-8035-4e99-a59e-60270d18e78a/kube-state-metrics/0.log"
Jul 23 16:37:06.946700 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:06.946665    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-6b857_e46ca500-8035-4e99-a59e-60270d18e78a/kube-rbac-proxy-main/0.log"
Jul 23 16:37:06.971675 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:06.971642    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-6b857_e46ca500-8035-4e99-a59e-60270d18e78a/kube-rbac-proxy-self/0.log"
Jul 23 16:37:07.009076 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.008987    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-76785b994f-zdt2n_e735cb6a-9f17-4603-9ac3-06f64f8d88c6/metrics-server/0.log"
Jul 23 16:37:07.033956 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.033866    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-dmcs8_75cf7d4c-9cec-4f95-ba6a-9934fbb8221e/monitoring-plugin/0.log"
Jul 23 16:37:07.221372 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.221340    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sc4r8_35f95106-7910-4fd8-9f81-2f9c2ac2d0f7/node-exporter/0.log"
Jul 23 16:37:07.244227 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.244192    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sc4r8_35f95106-7910-4fd8-9f81-2f9c2ac2d0f7/kube-rbac-proxy/0.log"
Jul 23 16:37:07.267820 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.267773    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sc4r8_35f95106-7910-4fd8-9f81-2f9c2ac2d0f7/init-textfile/0.log"
Jul 23 16:37:07.298791 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.298697    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-gq2vk_ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e/kube-rbac-proxy-main/0.log"
Jul 23 16:37:07.324816 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.324754    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-gq2vk_ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e/kube-rbac-proxy-self/0.log"
Jul 23 16:37:07.354692 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.354658    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-gq2vk_ad4e55e7-b5c3-4d0b-9eee-ccec23b7f30e/openshift-state-metrics/0.log"
Jul 23 16:37:07.396378 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.396345    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0600ef0a-4537-45d5-a61a-a3bef7fc6323/prometheus/0.log"
Jul 23 16:37:07.420184 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.420154    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0600ef0a-4537-45d5-a61a-a3bef7fc6323/config-reloader/0.log"
Jul 23 16:37:07.442872 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.442841    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0600ef0a-4537-45d5-a61a-a3bef7fc6323/thanos-sidecar/0.log"
Jul 23 16:37:07.466290 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.466136    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0600ef0a-4537-45d5-a61a-a3bef7fc6323/kube-rbac-proxy-web/0.log"
Jul 23 16:37:07.490662 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.490633    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0600ef0a-4537-45d5-a61a-a3bef7fc6323/kube-rbac-proxy/0.log"
Jul 23 16:37:07.519996 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.519963    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0600ef0a-4537-45d5-a61a-a3bef7fc6323/kube-rbac-proxy-thanos/0.log"
Jul 23 16:37:07.542178 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.542146    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0600ef0a-4537-45d5-a61a-a3bef7fc6323/init-config-reloader/0.log"
Jul 23 16:37:07.577716 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.577634    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-qp7gz_b84cd16a-1a1b-4672-8269-95064802a81e/prometheus-operator/0.log"
Jul 23 16:37:07.596998 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.596960    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-qp7gz_b84cd16a-1a1b-4672-8269-95064802a81e/kube-rbac-proxy/0.log"
Jul 23 16:37:07.621871 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.621843    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-r65r6_1079e788-8486-467b-b624-d97bec73a90a/prometheus-operator-admission-webhook/0.log"
Jul 23 16:37:07.660251 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.660213    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-db958c45b-28plx_af6a8538-6dc9-435b-94eb-eae497220f3c/telemeter-client/0.log"
Jul 23 16:37:07.689004 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.688981    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-db958c45b-28plx_af6a8538-6dc9-435b-94eb-eae497220f3c/reload/0.log"
Jul 23 16:37:07.723509 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.723481    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-db958c45b-28plx_af6a8538-6dc9-435b-94eb-eae497220f3c/kube-rbac-proxy/0.log"
Jul 23 16:37:07.755274 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.755208    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d8984d499-4m7rc_5c1bc9a7-99f1-4dba-b6b1-cc797948f674/thanos-query/0.log"
Jul 23 16:37:07.782511 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.782487    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d8984d499-4m7rc_5c1bc9a7-99f1-4dba-b6b1-cc797948f674/kube-rbac-proxy-web/0.log"
Jul 23 16:37:07.816329 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.816297    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d8984d499-4m7rc_5c1bc9a7-99f1-4dba-b6b1-cc797948f674/kube-rbac-proxy/0.log"
Jul 23 16:37:07.838463 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.838385    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d8984d499-4m7rc_5c1bc9a7-99f1-4dba-b6b1-cc797948f674/prom-label-proxy/0.log"
Jul 23 16:37:07.864284 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.864252    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d8984d499-4m7rc_5c1bc9a7-99f1-4dba-b6b1-cc797948f674/kube-rbac-proxy-rules/0.log"
Jul 23 16:37:07.890784 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:07.890760    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d8984d499-4m7rc_5c1bc9a7-99f1-4dba-b6b1-cc797948f674/kube-rbac-proxy-metrics/0.log"
Jul 23 16:37:10.112427 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:10.112404    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-77dfddf987-rqlwq_e38d3077-86da-4f74-852b-4f84fe4c8a74/console/0.log"
Jul 23 16:37:10.146920 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:10.146888    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-hbd4t_a1ee8604-fbac-48a8-96d7-2d8d9648cd05/download-server/0.log"
Jul 23 16:37:11.747291 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.747252    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gvkpg_0074ed98-eb79-4d56-9359-0dbf13b31760/dns/0.log"
Jul 23 16:37:11.769964 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.769926    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gvkpg_0074ed98-eb79-4d56-9359-0dbf13b31760/kube-rbac-proxy/0.log"
Jul 23 16:37:11.815658 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.815621    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"]
Jul 23 16:37:11.823439 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.823404    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.829731 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.829700    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"]
Jul 23 16:37:11.895572 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.895528    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-proc\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.895767 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.895697    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-lib-modules\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.895767 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.895756    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-sys\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.895917 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.895782    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rjmj\" (UniqueName: \"kubernetes.io/projected/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-kube-api-access-9rjmj\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.895981 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.895913    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-podres\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.930702 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.930669    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-564fg_cae772ab-5fcb-444a-b8c9-79e8048348e1/dns-node-resolver/0.log"
Jul 23 16:37:11.996759 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.996719    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-podres\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.996946 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.996769    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-proc\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.996946 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.996894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-lib-modules\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.996946 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.996908    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-podres\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.996946 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.996925    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-proc\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.996946 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.996933    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-sys\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.997160 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.996979    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-sys\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.997160 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.997013    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-lib-modules\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:11.997160 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:11.996978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rjmj\" (UniqueName: \"kubernetes.io/projected/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-kube-api-access-9rjmj\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:12.005835 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:12.005743    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rjmj\" (UniqueName: \"kubernetes.io/projected/6caf5553-cd51-4588-8e5b-a7b4c284f2a8-kube-api-access-9rjmj\") pod \"perf-node-gather-daemonset-52f6t\" (UID: \"6caf5553-cd51-4588-8e5b-a7b4c284f2a8\") " pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:12.136636 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:12.136581    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:12.310366 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:12.310257    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"]
Jul 23 16:37:12.314491 ip-10-0-140-230 kubenswrapper[2562]: W0723 16:37:12.314451    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6caf5553_cd51_4588_8e5b_a7b4c284f2a8.slice/crio-c7062884e79ed1fcf312a6327555354f01ba930b7b811262d2ddaca772893c7b WatchSource:0}: Error finding container c7062884e79ed1fcf312a6327555354f01ba930b7b811262d2ddaca772893c7b: Status 404 returned error can't find the container with id c7062884e79ed1fcf312a6327555354f01ba930b7b811262d2ddaca772893c7b
Jul 23 16:37:12.566255 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:12.566173    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p995g_f1a0d25d-5f09-4812-9861-7678dc1af972/node-ca/0.log"
Jul 23 16:37:12.710734 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:12.710690    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t" event={"ID":"6caf5553-cd51-4588-8e5b-a7b4c284f2a8","Type":"ContainerStarted","Data":"8455af9fd7dee2dc1d268aadb12880bccc55c487a1c7239da9825267fe9dec6e"}
Jul 23 16:37:12.711081 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:12.711059    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t" event={"ID":"6caf5553-cd51-4588-8e5b-a7b4c284f2a8","Type":"ContainerStarted","Data":"c7062884e79ed1fcf312a6327555354f01ba930b7b811262d2ddaca772893c7b"}
Jul 23 16:37:12.712046 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:12.712018    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:12.728217 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:12.728155    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t" podStartSLOduration=1.728134286 podStartE2EDuration="1.728134286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 16:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:37:12.727408323 +0000 UTC m=+2895.931775730" watchObservedRunningTime="2026-07-23 16:37:12.728134286 +0000 UTC m=+2895.932501698"
Jul 23 16:37:13.702414 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:13.702387    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fb5dcd56-rwpjm_7158c456-141f-4e29-8ce3-711b500e2b1a/kube-auth-proxy/0.log"
Jul 23 16:37:14.561179 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:14.561143    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-695p5_6e24b669-70a0-4855-8cbe-a476295a950a/serve-healthcheck-canary/0.log"
Jul 23 16:37:15.104766 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:15.104736    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-979hc_e31c4db0-378a-4f21-aa9e-24f4ed93743a/kube-rbac-proxy/0.log"
Jul 23 16:37:15.125501 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:15.125472    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-979hc_e31c4db0-378a-4f21-aa9e-24f4ed93743a/exporter/0.log"
Jul 23 16:37:15.147588 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:15.147560    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-979hc_e31c4db0-378a-4f21-aa9e-24f4ed93743a/extractor/0.log"
Jul 23 16:37:17.900056 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:17.900022    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8498cdcf8c-dws5b_0dbaf1a1-09be-4db8-92fe-d1a17ed4437e/manager/0.log"
Jul 23 16:37:18.223175 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:18.223139    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-646b5d5c79-tdxr9_d871218f-5819-44e3-9009-d2a196e35031/manager/0.log"
Jul 23 16:37:18.241334 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:18.241306    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-646b5d5c79-tdxr9_d871218f-5819-44e3-9009-d2a196e35031/copy-manifests/0.log"
Jul 23 16:37:19.419453 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:19.419419    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c5865ccbd-gph44_6c8d0ae6-dd1b-4844-9f13-8dcd175d7373/manager/0.log"
Jul 23 16:37:19.443655 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:19.443628    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fb5dcd56-rwpjm_7158c456-141f-4e29-8ce3-711b500e2b1a/kube-auth-proxy/0.log"
Jul 23 16:37:19.445028 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:19.444979    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-vpmmp_1295e835-f6c9-4666-8598-c364388eb6b2/openshift-lws-operator/0.log"
Jul 23 16:37:19.736517 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:19.736441    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4nhtn/perf-node-gather-daemonset-52f6t"
Jul 23 16:37:21.041462 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:21.041374    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fb5dcd56-rwpjm_7158c456-141f-4e29-8ce3-711b500e2b1a/kube-auth-proxy/0.log"
Jul 23 16:37:26.165282 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:26.165250    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-57vf5_ad0c93e9-d754-43b4-9310-125f0fe25a9e/kube-multus/0.log"
Jul 23 16:37:26.187743 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:26.187718    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97djp_07e7e5bd-7e31-41af-8981-875038b497bc/kube-multus-additional-cni-plugins/0.log"
Jul 23 16:37:26.207466 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:26.207443    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97djp_07e7e5bd-7e31-41af-8981-875038b497bc/egress-router-binary-copy/0.log"
Jul 23 16:37:26.226937 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:26.226911    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97djp_07e7e5bd-7e31-41af-8981-875038b497bc/cni-plugins/0.log"
Jul 23 16:37:26.249194 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:26.249153    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97djp_07e7e5bd-7e31-41af-8981-875038b497bc/bond-cni-plugin/0.log"
Jul 23 16:37:26.273555 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:26.273520    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97djp_07e7e5bd-7e31-41af-8981-875038b497bc/routeoverride-cni/0.log"
Jul 23 16:37:26.294300 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:26.294278    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97djp_07e7e5bd-7e31-41af-8981-875038b497bc/whereabouts-cni-bincopy/0.log"
Jul 23 16:37:26.316368 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:26.316347    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97djp_07e7e5bd-7e31-41af-8981-875038b497bc/whereabouts-cni/0.log"
Jul 23 16:37:26.832824 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:26.832783    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cnjpz_222a0a15-7ded-4543-94e0-21ddcd1bcf3d/network-metrics-daemon/0.log"
Jul 23 16:37:26.850996 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:26.850969    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cnjpz_222a0a15-7ded-4543-94e0-21ddcd1bcf3d/kube-rbac-proxy/0.log"
Jul 23 16:37:28.028838 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:28.028794    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-controller/0.log"
Jul 23 16:37:28.044147 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:28.044118    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/1.log"
Jul 23 16:37:28.044299 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:28.044138    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovn-acl-logging/0.log"
Jul 23 16:37:28.060487 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:28.060458    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/kube-rbac-proxy-node/0.log"
Jul 23 16:37:28.081742 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:28.081720    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/kube-rbac-proxy-ovn-metrics/0.log"
Jul 23 16:37:28.101896 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:28.101870    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/northd/0.log"
Jul 23 16:37:28.122986 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:28.122965    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/nbdb/0.log"
Jul 23 16:37:28.149970 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:28.149941    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/sbdb/0.log"
Jul 23 16:37:28.216428 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:28.216400    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9x5jr_d14e04e1-071c-4b0e-8a00-188ba9797d1a/ovnkube-controller/0.log"
Jul 23 16:37:29.638371 ip-10-0-140-230 kubenswrapper[2562]: I0723 16:37:29.638338    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-zhhqf_1fd76b11-e5a8-43fa-b7d8-aa2b8a3ba441/network-check-target-container/0.log"