Jul 06 19:50:19.556494 ip-10-0-139-90 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 19:50:20.017227 ip-10-0-139-90 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 19:50:20.017227 ip-10-0-139-90 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 19:50:20.017227 ip-10-0-139-90 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 19:50:20.017227 ip-10-0-139-90 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 19:50:20.017227 ip-10-0-139-90 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 19:50:20.019209 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.019058    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 19:50:20.025913 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025893    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:50:20.025913 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025913    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025917    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025921    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025925    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025928    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025931    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025934    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025937    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025940    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025943    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025946    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025949    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025951    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025961    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025964    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025967    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025970    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025973    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025975    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025978    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:50:20.025982 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025982    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025985    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025988    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025991    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025994    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.025998    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026001    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026004    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026007    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026010    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026013    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026015    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026019    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026022    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026024    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026027    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026030    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026032    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026035    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026037    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:50:20.026507 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026040    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026043    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026046    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026048    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026051    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026054    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026057    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026061    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026064    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026067    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026070    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026073    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026075    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026078    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026082    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026085    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026088    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026091    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026094    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:50:20.027042 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026097    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026099    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026103    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026105    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026108    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026111    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026114    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026117    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026120    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026122    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026125    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026128    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026130    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026133    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026136    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026138    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026141    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026144    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026148    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026150    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026153    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:50:20.027566 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026156    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026159    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026163    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026167    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026170    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026173    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026176    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026179    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026182    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026184    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026189    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026193    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026196    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026200    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026204    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026208    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026211    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.026214    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:50:20.028108 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027028    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027038    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027041    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027045    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027048    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027051    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027054    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027057    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027060    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027063    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027066    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027068    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027071    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027074    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027078    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027081    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027083    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027087    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027090    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027092    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:50:20.028619 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027095    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027098    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027101    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027104    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027106    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027109    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027112    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027114    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027117    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027120    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027122    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027125    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027128    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027131    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027133    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027136    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027139    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027141    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027144    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027147    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:50:20.029146 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027150    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027153    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027156    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027158    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027161    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027164    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027167    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027170    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027173    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027176    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027179    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027181    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027186    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027189    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027192    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027194    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027197    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027200    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027203    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027205    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:50:20.029682 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027209    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027211    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027214    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027217    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027219    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027222    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027225    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027228    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027230    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027233    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027236    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027238    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027241    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027244    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027262    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027267    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027270    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027273    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027276    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027279    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:50:20.030199 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027282    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027287    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027292    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027295    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027306    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027309    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027312    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027316    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027319    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027323    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027327    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027330    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027332    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027335    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027338    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027341    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027344    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027347    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:50:20.030728 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027349    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027829    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027835    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027839    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027842    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027845    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027848    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027851    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027854    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027857    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027859    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027865    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027868    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027870    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027874    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027876    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027879    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027882    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027886    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:50:20.031222 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027890    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027894    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027897    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027900    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027903    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027906    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027909    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027911    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027914    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027917    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027920    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027923    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027926    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027929    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027931    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027934    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027937    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027939    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027942    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027945    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:50:20.031749 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027947    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027950    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027953    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027956    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027960    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027963    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027966    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027968    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027972    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027975    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027978    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027981    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027984    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027988    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027990    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027993    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027995    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.027998    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028001    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028004    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:50:20.032287 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028007    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028009    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028012    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028015    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028018    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028021    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028024    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028026    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028029    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028031    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028034    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028037    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028039    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028042    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028044    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028048    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028052    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028054    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028057    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028060    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:50:20.032875 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028063    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028066    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028069    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028071    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028074    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028078    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028082    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028084    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028087    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028090    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028092    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028095    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028098    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028100    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028103    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028106    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028109    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028112    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028116    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:50:20.033423 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028120    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028122    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028196    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028203    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028211    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028216    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028220    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028224    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028229    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028234    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028238    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028241    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028245    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028264    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028268    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028272    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028275    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028278    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028281    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028284    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028289    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028292    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028295    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028298    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 19:50:20.033923 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028302    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028306    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028309    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028313    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028316    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028319    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028322    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028325    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028329    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028332    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028336    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028339    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028342    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028346    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028349    2577 flags.go:64] FLAG: --enable-server="true"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028354    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028359    2577 flags.go:64] FLAG: --event-burst="100"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028363    2577 flags.go:64] FLAG: --event-qps="50"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028366    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028369    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028373    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028381    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028384    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028387    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028391    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 06 19:50:20.034600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028395    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028398    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028401    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028404    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028407    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028411    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028414    2577 flags.go:64] FLAG: --feature-gates=""
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028418    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028421    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028425    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028429    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028432    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028435    2577 flags.go:64] FLAG: --help="false"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028438    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028442    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028445    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028448    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028452    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028455    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028458    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028461    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028464    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028468    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 19:50:20.035262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028472    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028476    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028479    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028482    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028485    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028488    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028491    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028494    2577 flags.go:64] FLAG: --lock-file=""
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028497    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028501    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028504    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028546    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028554    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028559    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028563    2577 flags.go:64] FLAG: --logging-format="text"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028568    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028576    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028579    2577 flags.go:64] FLAG: --manifest-url=""
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028583    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028590    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028594    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028600    2577 flags.go:64] FLAG: --max-pods="110"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028603    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028607    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028610    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 19:50:20.035903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028614    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028618    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028621    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028625    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028632    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028636    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028639    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028643    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028646    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028653    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028656    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028660    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028663    2577 flags.go:64] FLAG: --port="10250"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028666    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028669    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dadb8b971f6d3eb5"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028673    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028677    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028680    2577 flags.go:64] FLAG: --register-node="true"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028684    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028689    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028693    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028696    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028699    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028703    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028707    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 19:50:20.036768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028710    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028713    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028717    2577 flags.go:64] FLAG: --runonce="false"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028720    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028724    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028727    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028730    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028733    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028737    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028740    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028743    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028747    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028750    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028753    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028756    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028759    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028762    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028766    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028772    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028775    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028779    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028785    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028788    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028791    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028794    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 19:50:20.037712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028801    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028805    2577 flags.go:64] FLAG: --v="2"
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028810    2577 flags.go:64] FLAG: --version="false"
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028814    2577 flags.go:64] FLAG: --vmodule=""
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028818    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.028822    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028969    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028973    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028976    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028979    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028982    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028985    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028988    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028991    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028993    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028996    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.028999    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029002    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029007    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029010    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029014    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:50:20.038395 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029017    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029020    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029024    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029026    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029030    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029033    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029036    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029039    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029041    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029044    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029047    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029053    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029056    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029058    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029061    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029064    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029067    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029070    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029072    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029075    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:50:20.038976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029078    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029081    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029086    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029089    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029093    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029096    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029099    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029102    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029105    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029109    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029112    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029115    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029118    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029121    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029123    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029126    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029129    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029132    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029134    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:50:20.039521 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029137    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029140    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029143    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029145    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029150    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029153    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029156    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029159    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029163    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029166    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029169    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029172    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029175    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029178    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029180    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029183    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029186    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029188    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029191    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029194    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:50:20.040025 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029197    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029199    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029202    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029205    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029208    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029211    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029213    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029216    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029219    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029223    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029242    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029246    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029249    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029252    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029255    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029258    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029263    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029266    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029269    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:50:20.040820 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029272    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029275    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029278    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029281    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029283    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.029286    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.029981    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.038716    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.038736    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.038757    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.038764    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038827    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038833    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038836    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:50:20.041338 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038840    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038843    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038846    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038849    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038852    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038855    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038858    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038861    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038864    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038867    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038869    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038874    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038878    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038881    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038884    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038888    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038892    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038895    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038904    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:50:20.041725 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038907    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038910    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038912    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038915    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038918    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038921    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038924    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038927    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038929    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038932    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038935    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038938    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038941    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038943    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038946    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038949    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038952    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038955    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038958    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038960    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:50:20.042244 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038965    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038969    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038972    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038976    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038980    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038984    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038987    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038990    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038993    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038996    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.038999    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039002    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039005    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039007    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039010    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039013    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039015    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039018    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039021    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:50:20.042790 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039023    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039026    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039029    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039032    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039035    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039038    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039040    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039043    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039046    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039048    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039051    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039054    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039056    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039059    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039062    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039065    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039068    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039071    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039075    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039078    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:50:20.043308 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039081    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039083    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039086    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039089    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039092    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039094    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039097    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039100    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039102    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039105    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039108    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039110    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039113    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039116    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039118    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039121    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039123    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:50:20.043833 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039126    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.039131    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039278    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039286    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039291    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039295    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039298    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039301    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039305    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039308    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039310    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039313    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039316    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039319    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 19:50:20.044376 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039323    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039327    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039330    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039333    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039335    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039338    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039341    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039343    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039347    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039350    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039352    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039355    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039358    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039361    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039363    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039366    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039369    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039371    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039375    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 19:50:20.044773 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039379    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039383    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039387    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039391    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039394    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039397    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039401    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039404    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039406    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039409    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039413    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039415    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039418    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039421    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039424    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039427    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039430    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039433    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039436    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 19:50:20.045262 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039439    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039441    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039444    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039447    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039449    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039452    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039455    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039458    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039460    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039463    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039465    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039468    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039471    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039474    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039476    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039479    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039482    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039484    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039487    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039490    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 19:50:20.045756 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039492    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039495    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039498    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039500    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039503    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039506    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039509    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039512    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039515    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039518    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039521    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039524    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039526    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039529    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039532    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039535    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039538    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039541    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039544    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039546    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 19:50:20.046285 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039549    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039551    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039554    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039557    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039559    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039562    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039565    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039567    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:20.039570    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.039576    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.039699    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.041949    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.043710    2577 server.go:1019] "Starting client certificate rotation"
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.043821    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 19:50:20.046830 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.043856    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 19:50:20.047212 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.046652    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 19:50:20.052224 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.052203    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 19:50:20.067295 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.067271    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 06 19:50:20.069875 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.069854    2577 log.go:25] "Validated CRI v1 image API"
Jul 06 19:50:20.071617 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.071593    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 19:50:20.074843 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.074818    2577 fs.go:135] Filesystem UUIDs: map[1df1e02e-fee5-4b0d-a5d0-232fb1b08326:/dev/nvme0n1p4 63f752d7-f61e-42d5-8ac4-4ef697aca641:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 06 19:50:20.074938 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.074842    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 19:50:20.075521 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.075502    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 19:50:20.083047 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.082911    2577 manager.go:216] Machine: {Timestamp:2026-07-06 19:50:20.080917293 +0000 UTC m=+0.392372509 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099765 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e6336e9648113396ee5e6401b3601 SystemUUID:ec2e6336-e964-8113-396e-e5e6401b3601 BootID:ecf01487-1549-4413-9fa0-9f21a768331a Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9f:13:30:6f:97 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9f:13:30:6f:97 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:3a:b8:00:d7:88 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 19:50:20.083047 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.083027    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 19:50:20.083253 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.083158    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 19:50:20.088120 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.088086    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 19:50:20.088291 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.088122    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-90.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 19:50:20.089069 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.089058    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 19:50:20.089105 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.089071    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 19:50:20.089105 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.089079    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 19:50:20.089105 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.089097    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 19:50:20.091032 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.091018    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 19:50:20.094984 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.094936    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 19:50:20.095078 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.095067    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 19:50:20.095124 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.095102    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 19:50:20.095124 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.095113    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 19:50:20.095204 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.095128    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 19:50:20.095204 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.095138    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 06 19:50:20.095204 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.095149    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 19:50:20.095522 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.095505    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 19:50:20.095580 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.095532    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 19:50:20.097524 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.097498    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kjw8h"
Jul 06 19:50:20.098013 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.097992    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 19:50:20.100483 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.100467    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 19:50:20.100551 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.100500    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 19:50:20.101851 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101839    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 19:50:20.101917 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101856    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 19:50:20.101917 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101863    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 19:50:20.101917 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101870    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 19:50:20.101917 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101880    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 19:50:20.101917 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101886    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 19:50:20.101917 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101895    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 19:50:20.101917 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101901    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 19:50:20.101917 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101909    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 19:50:20.101917 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101915    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 19:50:20.102183 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101924    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 19:50:20.102183 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.101930    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 19:50:20.103727 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.103713    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 19:50:20.103727 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.103723    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 19:50:20.106618 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.106593    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kjw8h"
Jul 06 19:50:20.107104 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.107086    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-90.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 19:50:20.107928 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.107903    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-90.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 19:50:20.108002 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.107905    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 19:50:20.108556 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.108536    2577 server.go:1268] "Started kubelet"
Jul 06 19:50:20.108661 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.108623    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 19:50:20.109480 ip-10-0-139-90 systemd[1]: Started Kubernetes Kubelet.
Jul 06 19:50:20.110047 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.109985    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 19:50:20.110102 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.110059    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 19:50:20.111487 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.111470    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 19:50:20.113046 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.113028    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 19:50:20.114208 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.114184    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 19:50:20.115464 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.114360    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-90.ec2.internal.18bfcb384b927073  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-90.ec2.internal,UID:ip-10-0-139-90.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-90.ec2.internal,},FirstTimestamp:2026-07-06 19:50:20.108501107 +0000 UTC m=+0.419956322,LastTimestamp:2026-07-06 19:50:20.108501107 +0000 UTC m=+0.419956322,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-90.ec2.internal,}"
Jul 06 19:50:20.118961 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.118945    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 19:50:20.119012 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.118968    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 19:50:20.119078 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.119055    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:20.119764 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.119743    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 19:50:20.119764 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.119757    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 19:50:20.119895 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.119774    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 19:50:20.119895 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.119832    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 19:50:20.119895 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.119840    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 19:50:20.120051 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.119925    2577 factory.go:55] Registering systemd factory
Jul 06 19:50:20.120051 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.119953    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 06 19:50:20.120253 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.120235    2577 factory.go:153] Registering CRI-O factory
Jul 06 19:50:20.120253 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.120255    2577 factory.go:223] Registration of the crio container factory successfully
Jul 06 19:50:20.120391 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.120301    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 19:50:20.120391 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.120324    2577 factory.go:103] Registering Raw factory
Jul 06 19:50:20.120391 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.120340    2577 manager.go:1196] Started watching for new ooms in manager
Jul 06 19:50:20.120897 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.120875    2577 manager.go:349] Starting recovery of all containers
Jul 06 19:50:20.130377 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.130351    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:50:20.130543 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.130510    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 19:50:20.133794 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.133771    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-90.ec2.internal\" not found" node="ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.134947 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.134927    2577 manager.go:354] Recovery completed
Jul 06 19:50:20.141313 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.141166    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:50:20.143436 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.143419    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:50:20.143505 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.143459    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:50:20.143505 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.143474    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:50:20.144069 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.144054    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 19:50:20.144069 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.144061    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:50:20.144227 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.144070    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 19:50:20.144227 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.144093    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 19:50:20.144853 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.144838    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:50:20.144908 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.144865    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:50:20.144908 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.144876    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:50:20.145652 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.145639    2577 policy_none.go:49] "None policy: Start"
Jul 06 19:50:20.145698 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.145656    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 19:50:20.145698 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.145667    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 19:50:20.146712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.146701    2577 policy_none.go:47] "Start"
Jul 06 19:50:20.178863 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.178846    2577 manager.go:337] "Starting Device Plugin manager"
Jul 06 19:50:20.178970 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.178881    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 19:50:20.178970 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.178894    2577 server.go:87] "Starting device plugin registration server"
Jul 06 19:50:20.179208 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.179195    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 19:50:20.179245 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.179213    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 19:50:20.179328 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.179309    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 19:50:20.179463 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.179442    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 19:50:20.179463 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.179460    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 19:50:20.180021 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.179999    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 19:50:20.180096 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.180061    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:20.279551 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.279495    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:50:20.280880 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.280673    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:50:20.280880 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.280707    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:50:20.280880 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.280720    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:50:20.280880 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.280752    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.287406 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.287384    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.287500 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.287419    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-90.ec2.internal\": node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:20.302529 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.302506    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:20.331495 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.331462    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 19:50:20.331625 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.331513    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 19:50:20.331625 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.331540    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 19:50:20.331625 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.331586    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 19:50:20.333981 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.333955    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:50:20.404248 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.404212    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:20.431710 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.431679    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-90.ec2.internal"]
Jul 06 19:50:20.431804 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.431761    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:50:20.432848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.432828    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:50:20.432943 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.432862    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:50:20.432943 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.432873    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:50:20.434344 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.434331    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:50:20.434482 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.434458    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.434524 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.434499    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:50:20.435144 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.435131    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:50:20.435236 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.435158    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:50:20.435236 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.435170    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:50:20.435523 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.435508    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:50:20.435588 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.435542    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:50:20.435588 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.435558    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:50:20.436628 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.436613    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.436705 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.436638    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 19:50:20.437300 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.437285    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 19:50:20.437404 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.437318    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 19:50:20.437404 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.437333    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeHasSufficientPID"
Jul 06 19:50:20.463161 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.463137    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-90.ec2.internal\" not found" node="ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.467974 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.467956    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-90.ec2.internal\" not found" node="ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.504740 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.504716    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:20.521932 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.521908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4701a56854965b6d68e38c039cde7250-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal\" (UID: \"4701a56854965b6d68e38c039cde7250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.522018 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.521943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4701a56854965b6d68e38c039cde7250-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal\" (UID: \"4701a56854965b6d68e38c039cde7250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.522018 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.521969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/36e6979a7a16f66ce876f6ff5066432c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-90.ec2.internal\" (UID: \"36e6979a7a16f66ce876f6ff5066432c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.605274 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.605230    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:20.622904 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.622870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4701a56854965b6d68e38c039cde7250-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal\" (UID: \"4701a56854965b6d68e38c039cde7250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.622989 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.622924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4701a56854965b6d68e38c039cde7250-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal\" (UID: \"4701a56854965b6d68e38c039cde7250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.622989 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.622944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/36e6979a7a16f66ce876f6ff5066432c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-90.ec2.internal\" (UID: \"36e6979a7a16f66ce876f6ff5066432c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.622989 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.622970    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/36e6979a7a16f66ce876f6ff5066432c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-90.ec2.internal\" (UID: \"36e6979a7a16f66ce876f6ff5066432c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.622989 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.622874    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4701a56854965b6d68e38c039cde7250-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal\" (UID: \"4701a56854965b6d68e38c039cde7250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.623137 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.623032    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4701a56854965b6d68e38c039cde7250-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal\" (UID: \"4701a56854965b6d68e38c039cde7250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.706414 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.706337    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:20.769613 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.769587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.772272 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:20.772251    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:20.806912 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.806875    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:20.908170 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:20.908130    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:21.009338 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:21.009228    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:21.034456 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.034423    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:50:21.043227 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.043203    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 19:50:21.043370 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.043350    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 19:50:21.043417 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.043390    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 19:50:21.043417 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.043392    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 19:50:21.109918 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:21.109889    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:21.115607 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.115566    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 19:45:20 +0000 UTC" deadline="2028-01-10 14:58:08.435201449 +0000 UTC"
Jul 06 19:50:21.115681 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.115607    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13267h7m47.319596845s"
Jul 06 19:50:21.210355 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:21.210323    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:21.310730 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:21.310706    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:21.313678 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:21.313641    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e6979a7a16f66ce876f6ff5066432c.slice/crio-003bd2d6f90010a3c1a790fa4b3d793032f473446a115f67ab9b10dcaecce5fa WatchSource:0}: Error finding container 003bd2d6f90010a3c1a790fa4b3d793032f473446a115f67ab9b10dcaecce5fa: Status 404 returned error can't find the container with id 003bd2d6f90010a3c1a790fa4b3d793032f473446a115f67ab9b10dcaecce5fa
Jul 06 19:50:21.314223 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:21.314200    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4701a56854965b6d68e38c039cde7250.slice/crio-534d848e4fb7a3a6ebf8467c66a54c502b75ceee3785341f7b723382bcf22941 WatchSource:0}: Error finding container 534d848e4fb7a3a6ebf8467c66a54c502b75ceee3785341f7b723382bcf22941: Status 404 returned error can't find the container with id 534d848e4fb7a3a6ebf8467c66a54c502b75ceee3785341f7b723382bcf22941
Jul 06 19:50:21.318850 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.318835    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 19:50:21.334758 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.334707    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-90.ec2.internal" event={"ID":"36e6979a7a16f66ce876f6ff5066432c","Type":"ContainerStarted","Data":"003bd2d6f90010a3c1a790fa4b3d793032f473446a115f67ab9b10dcaecce5fa"}
Jul 06 19:50:21.335871 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.335844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal" event={"ID":"4701a56854965b6d68e38c039cde7250","Type":"ContainerStarted","Data":"534d848e4fb7a3a6ebf8467c66a54c502b75ceee3785341f7b723382bcf22941"}
Jul 06 19:50:21.415493 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:21.415459    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:21.521908 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:21.521863    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:21.622598 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:21.622573    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-90.ec2.internal\" not found"
Jul 06 19:50:21.699604 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.699570    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:50:21.719235 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.719174    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:21.735125 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.735099    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 19:50:21.736302 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.736288    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-90.ec2.internal"
Jul 06 19:50:21.747659 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:21.747641    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 19:50:22.097056 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.096975    2577 apiserver.go:52] "Watching apiserver"
Jul 06 19:50:22.107540 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.107504    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 19:50:22.107972 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.107942    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-8x5db","kube-system/konnectivity-agent-x9qff","kube-system/kube-apiserver-proxy-ip-10-0-139-90.ec2.internal","openshift-cluster-node-tuning-operator/tuned-hbbkv","openshift-dns/node-resolver-qzw6h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal","openshift-multus/multus-additional-cni-plugins-8f42z","openshift-multus/multus-mbg4d","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl","openshift-image-registry/node-ca-29qdv","openshift-multus/network-metrics-daemon-6dbm7","openshift-network-diagnostics/network-check-target-vqvgv","openshift-network-operator/iptables-alerter-mms6t","openshift-ovn-kubernetes/ovnkube-node-q6sq5"]
Jul 06 19:50:22.110746 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.110706    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.113871 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.113840    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nqg2x\""
Jul 06 19:50:22.113985 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.113872    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 19:50:22.114137 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.114119    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 19:50:22.114201 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.114163    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 19:50:22.114201 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.114170    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 19:50:22.114201 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.114179    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 19:50:22.114413 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.114391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mms6t"
Jul 06 19:50:22.116135 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.116105    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.116793 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.116774    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 19:50:22.118236 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.117964    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 19:50:22.118236 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.118006    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 19:50:22.118236 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.118029    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rz4d8\""
Jul 06 19:50:22.120077 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.120051    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.124091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.120445    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 19:50:22.124091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.121776    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 19:50:22.124091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.122489    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 19:50:22.124091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.122566    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-h7hgp\""
Jul 06 19:50:22.124091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.122824    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 19:50:22.124091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.123032    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 19:50:22.124091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.123208    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 19:50:22.124091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.123275    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 19:50:22.124091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.123457    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 19:50:22.124091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.123756    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 19:50:22.124091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.124063    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qscs7\""
Jul 06 19:50:22.124873 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.124477    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qzw6h"
Jul 06 19:50:22.126030 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.126005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.127686 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.127664    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 19:50:22.127984 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.127876    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 19:50:22.127984 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.127905    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jhwpc\""
Jul 06 19:50:22.129109 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.129088    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sl768\""
Jul 06 19:50:22.129830 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.129813    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 19:50:22.130084 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.130060    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.131686 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.131665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgdbl\" (UniqueName: \"kubernetes.io/projected/f2473042-4162-458d-8969-ae7e8668b0b3-kube-api-access-bgdbl\") pod \"node-resolver-qzw6h\" (UID: \"f2473042-4162-458d-8969-ae7e8668b0b3\") " pod="openshift-dns/node-resolver-qzw6h"
Jul 06 19:50:22.131799 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.131692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/882218e0-217a-4d4f-aff1-121417a806e7-ovnkube-config\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.131799 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.131709    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f2473042-4162-458d-8969-ae7e8668b0b3-hosts-file\") pod \"node-resolver-qzw6h\" (UID: \"f2473042-4162-458d-8969-ae7e8668b0b3\") " pod="openshift-dns/node-resolver-qzw6h"
Jul 06 19:50:22.131799 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.131733    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2473042-4162-458d-8969-ae7e8668b0b3-tmp-dir\") pod \"node-resolver-qzw6h\" (UID: \"f2473042-4162-458d-8969-ae7e8668b0b3\") " pod="openshift-dns/node-resolver-qzw6h"
Jul 06 19:50:22.131799 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.131768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-cni-bin\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.131799 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.131794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-os-release\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.132063 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.131821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6207540f-5268-4df4-b0fa-94cd2e4e4a44-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.132063 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.131847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-var-lib-openvswitch\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.132063 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.131872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-etc-openvswitch\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.132063 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.131896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-run-openvswitch\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.132063 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.131938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.132063 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132020    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-kubelet\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.132063 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-run-netns\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132067    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-run-systemd\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/882218e0-217a-4d4f-aff1-121417a806e7-ovn-node-metrics-cert\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132172    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/882218e0-217a-4d4f-aff1-121417a806e7-ovnkube-script-lib\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-sys\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132225    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-var-lib-kubelet\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbd2r\" (UniqueName: \"kubernetes.io/projected/47dc2bc5-2e8c-47a7-9a1e-5be37758f974-kube-api-access-xbd2r\") pod \"iptables-alerter-mms6t\" (UID: \"47dc2bc5-2e8c-47a7-9a1e-5be37758f974\") " pod="openshift-network-operator/iptables-alerter-mms6t"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132318    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/47dc2bc5-2e8c-47a7-9a1e-5be37758f974-iptables-alerter-script\") pod \"iptables-alerter-mms6t\" (UID: \"47dc2bc5-2e8c-47a7-9a1e-5be37758f974\") " pod="openshift-network-operator/iptables-alerter-mms6t"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132346    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-slash\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.132432 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-log-socket\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-sysconfig\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132472    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-systemd\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132516    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-lib-modules\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6207540f-5268-4df4-b0fa-94cd2e4e4a44-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24pk6\" (UniqueName: \"kubernetes.io/projected/6207540f-5268-4df4-b0fa-94cd2e4e4a44-kube-api-access-24pk6\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-modprobe-d\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132634    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-run\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132660    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-host\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132686    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132787    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-krnc9\""
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-tuned\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132900    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-cni-netd\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/189aff81-6dc1-4c79-b9c8-d6807cd546fb-tmp\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.133014 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.132998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-system-cni-dir\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47dc2bc5-2e8c-47a7-9a1e-5be37758f974-host-slash\") pod \"iptables-alerter-mms6t\" (UID: \"47dc2bc5-2e8c-47a7-9a1e-5be37758f974\") " pod="openshift-network-operator/iptables-alerter-mms6t"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133061    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-node-log\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133098    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/882218e0-217a-4d4f-aff1-121417a806e7-env-overrides\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzxfp\" (UniqueName: \"kubernetes.io/projected/882218e0-217a-4d4f-aff1-121417a806e7-kube-api-access-mzxfp\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.133131    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-kubernetes\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133253    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-sysctl-d\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-cnibin\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6207540f-5268-4df4-b0fa-94cd2e4e4a44-cni-binary-copy\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133401    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-systemd-units\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-run-ovn\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133447    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-sysctl-conf\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.133979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.133514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2sb9\" (UniqueName: \"kubernetes.io/projected/189aff81-6dc1-4c79-b9c8-d6807cd546fb-kube-api-access-d2sb9\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.135273 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.135076    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:22.137716 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.137691    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-29qdv"
Jul 06 19:50:22.138308 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.138180    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 19:50:22.138460 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.138441    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 19:50:22.138537 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.138523    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ptgp5\""
Jul 06 19:50:22.140472 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.140453    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 19:50:22.140932 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.140914    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 19:50:22.141310 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.141291    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 19:50:22.141547 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.141514    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rsgdq\""
Jul 06 19:50:22.141669 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.141653    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:22.141775 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.141751    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:22.144146 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.143985    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:22.144146 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.144050    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:22.160121 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.160091    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wgm5t"
Jul 06 19:50:22.168436 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.168414    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wgm5t"
Jul 06 19:50:22.221060 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.221028    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 19:50:22.234458 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbd2r\" (UniqueName: \"kubernetes.io/projected/47dc2bc5-2e8c-47a7-9a1e-5be37758f974-kube-api-access-xbd2r\") pod \"iptables-alerter-mms6t\" (UID: \"47dc2bc5-2e8c-47a7-9a1e-5be37758f974\") " pod="openshift-network-operator/iptables-alerter-mms6t"
Jul 06 19:50:22.234648 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-slash\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.234648 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-log-socket\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.234648 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-systemd\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.234648 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-device-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.234648 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-modprobe-d\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.234648 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-run\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.234648 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-host\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.234648 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-tuned\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-cni-netd\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/189aff81-6dc1-4c79-b9c8-d6807cd546fb-tmp\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-hostroot\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-kubelet-config\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234807    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-slash\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234821    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-systemd\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234812    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-cni-netd\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47dc2bc5-2e8c-47a7-9a1e-5be37758f974-host-slash\") pod \"iptables-alerter-mms6t\" (UID: \"47dc2bc5-2e8c-47a7-9a1e-5be37758f974\") " pod="openshift-network-operator/iptables-alerter-mms6t"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234821    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-host\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234871    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47dc2bc5-2e8c-47a7-9a1e-5be37758f974-host-slash\") pod \"iptables-alerter-mms6t\" (UID: \"47dc2bc5-2e8c-47a7-9a1e-5be37758f974\") " pod="openshift-network-operator/iptables-alerter-mms6t"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-kubernetes\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4b68573-84ea-4f3d-980c-882767d83501-agent-certs\") pod \"konnectivity-agent-x9qff\" (UID: \"f4b68573-84ea-4f3d-980c-882767d83501\") " pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234940    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-log-socket\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.234993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-socket-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-registration-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.235068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-etc-selinux\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235090    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6207540f-5268-4df4-b0fa-94cd2e4e4a44-cni-binary-copy\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-systemd-units\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235143    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-modprobe-d\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235154    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-run-netns\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-var-lib-cni-bin\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235213    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7m5z\" (UniqueName: \"kubernetes.io/projected/509ae1fb-ff23-4a78-acb5-5f55e5900408-kube-api-access-f7m5z\") pod \"node-ca-29qdv\" (UID: \"509ae1fb-ff23-4a78-acb5-5f55e5900408\") " pod="openshift-image-registry/node-ca-29qdv"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235227    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-run\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235236    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235338    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-kubernetes\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235403    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-systemd-units\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/882218e0-217a-4d4f-aff1-121417a806e7-ovnkube-config\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2473042-4162-458d-8969-ae7e8668b0b3-tmp-dir\") pod \"node-resolver-qzw6h\" (UID: \"f2473042-4162-458d-8969-ae7e8668b0b3\") " pod="openshift-dns/node-resolver-qzw6h"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce8dc260-f060-4662-9265-f8195b755570-cni-binary-copy\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-cni-bin\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-os-release\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.235848 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235614    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-cni-bin\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235680    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6207540f-5268-4df4-b0fa-94cd2e4e4a44-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-var-lib-openvswitch\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235745    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-os-release\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.235828    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-var-lib-openvswitch\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236215    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/882218e0-217a-4d4f-aff1-121417a806e7-ovnkube-config\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236232    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6207540f-5268-4df4-b0fa-94cd2e4e4a44-cni-binary-copy\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236246    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2473042-4162-458d-8969-ae7e8668b0b3-tmp-dir\") pod \"node-resolver-qzw6h\" (UID: \"f2473042-4162-458d-8969-ae7e8668b0b3\") " pod="openshift-dns/node-resolver-qzw6h"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-etc-openvswitch\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236393    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-etc-openvswitch\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236392    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6207540f-5268-4df4-b0fa-94cd2e4e4a44-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-run-openvswitch\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-sys\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236538    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-run-openvswitch\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-kubelet\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.236636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236622    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-kubelet\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236632    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-sys\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-run-netns\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236681    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/882218e0-217a-4d4f-aff1-121417a806e7-ovnkube-script-lib\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236703    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-run-netns\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-var-lib-kubelet\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-multus-cni-dir\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236778    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-cnibin\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-multus-conf-dir\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236862    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-var-lib-kubelet\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236863    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-etc-kubernetes\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/47dc2bc5-2e8c-47a7-9a1e-5be37758f974-iptables-alerter-script\") pod \"iptables-alerter-mms6t\" (UID: \"47dc2bc5-2e8c-47a7-9a1e-5be37758f974\") " pod="openshift-network-operator/iptables-alerter-mms6t"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.236982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-sysconfig\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-lib-modules\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/509ae1fb-ff23-4a78-acb5-5f55e5900408-host\") pod \"node-ca-29qdv\" (UID: \"509ae1fb-ff23-4a78-acb5-5f55e5900408\") " pod="openshift-image-registry/node-ca-29qdv"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:22.237321 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6207540f-5268-4df4-b0fa-94cd2e4e4a44-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24pk6\" (UniqueName: \"kubernetes.io/projected/6207540f-5268-4df4-b0fa-94cd2e4e4a44-kube-api-access-24pk6\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237231    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-lib-modules\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237214    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-run-k8s-cni-cncf-io\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-sys-fs\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-dbus\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khcf5\" (UniqueName: \"kubernetes.io/projected/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-kube-api-access-khcf5\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-run-multus-certs\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237407    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/882218e0-217a-4d4f-aff1-121417a806e7-ovnkube-script-lib\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-system-cni-dir\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237107    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-node-log\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237562    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-sysconfig\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/882218e0-217a-4d4f-aff1-121417a806e7-env-overrides\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237600    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-node-log\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzxfp\" (UniqueName: \"kubernetes.io/projected/882218e0-217a-4d4f-aff1-121417a806e7-kube-api-access-mzxfp\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-sysctl-d\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.238149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237747    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgdbl\" (UniqueName: \"kubernetes.io/projected/f2473042-4162-458d-8969-ae7e8668b0b3-kube-api-access-bgdbl\") pod \"node-resolver-qzw6h\" (UID: \"f2473042-4162-458d-8969-ae7e8668b0b3\") " pod="openshift-dns/node-resolver-qzw6h"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-var-lib-kubelet\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce8dc260-f060-4662-9265-f8195b755570-multus-daemon-config\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237875    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-cnibin\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-run-ovn\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237933    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-sysctl-conf\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.237958    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/47dc2bc5-2e8c-47a7-9a1e-5be37758f974-iptables-alerter-script\") pod \"iptables-alerter-mms6t\" (UID: \"47dc2bc5-2e8c-47a7-9a1e-5be37758f974\") " pod="openshift-network-operator/iptables-alerter-mms6t"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2sb9\" (UniqueName: \"kubernetes.io/projected/189aff81-6dc1-4c79-b9c8-d6807cd546fb-kube-api-access-d2sb9\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4b68573-84ea-4f3d-980c-882767d83501-konnectivity-ca\") pod \"konnectivity-agent-x9qff\" (UID: \"f4b68573-84ea-4f3d-980c-882767d83501\") " pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238084    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-system-cni-dir\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/509ae1fb-ff23-4a78-acb5-5f55e5900408-serviceca\") pod \"node-ca-29qdv\" (UID: \"509ae1fb-ff23-4a78-acb5-5f55e5900408\") " pod="openshift-image-registry/node-ca-29qdv"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238133    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6207540f-5268-4df4-b0fa-94cd2e4e4a44-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238164    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f2473042-4162-458d-8969-ae7e8668b0b3-hosts-file\") pod \"node-resolver-qzw6h\" (UID: \"f2473042-4162-458d-8969-ae7e8668b0b3\") " pod="openshift-dns/node-resolver-qzw6h"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238246    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-sysctl-conf\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-var-lib-cni-multus\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238329    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-run-ovn\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.238986 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238341    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f2473042-4162-458d-8969-ae7e8668b0b3-hosts-file\") pod \"node-resolver-qzw6h\" (UID: \"f2473042-4162-458d-8969-ae7e8668b0b3\") " pod="openshift-dns/node-resolver-qzw6h"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-os-release\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238463    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6207540f-5268-4df4-b0fa-94cd2e4e4a44-cnibin\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238100    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/882218e0-217a-4d4f-aff1-121417a806e7-env-overrides\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqqkj\" (UniqueName: \"kubernetes.io/projected/ce8dc260-f060-4662-9265-f8195b755570-kube-api-access-fqqkj\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238527    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgnf2\" (UniqueName: \"kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2\") pod \"network-check-target-vqvgv\" (UID: \"1b321be0-8e73-4d70-91d7-e42975820cb4\") " pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-run-systemd\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238632    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-sysctl-d\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238653    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-run-systemd\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238664    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238713    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/882218e0-217a-4d4f-aff1-121417a806e7-ovn-node-metrics-cert\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238789    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882218e0-217a-4d4f-aff1-121417a806e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-system-cni-dir\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-multus-socket-dir-parent\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.239856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.238895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tftct\" (UniqueName: \"kubernetes.io/projected/f658f919-d15f-4648-949f-fce06f96211c-kube-api-access-tftct\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.240649 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.240140    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/189aff81-6dc1-4c79-b9c8-d6807cd546fb-tmp\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.240714 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.240693    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/189aff81-6dc1-4c79-b9c8-d6807cd546fb-etc-tuned\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.241576 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.241554    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/882218e0-217a-4d4f-aff1-121417a806e7-ovn-node-metrics-cert\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.245767 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.245581    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbd2r\" (UniqueName: \"kubernetes.io/projected/47dc2bc5-2e8c-47a7-9a1e-5be37758f974-kube-api-access-xbd2r\") pod \"iptables-alerter-mms6t\" (UID: \"47dc2bc5-2e8c-47a7-9a1e-5be37758f974\") " pod="openshift-network-operator/iptables-alerter-mms6t"
Jul 06 19:50:22.248706 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.248203    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2sb9\" (UniqueName: \"kubernetes.io/projected/189aff81-6dc1-4c79-b9c8-d6807cd546fb-kube-api-access-d2sb9\") pod \"tuned-hbbkv\" (UID: \"189aff81-6dc1-4c79-b9c8-d6807cd546fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.248706 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.248631    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24pk6\" (UniqueName: \"kubernetes.io/projected/6207540f-5268-4df4-b0fa-94cd2e4e4a44-kube-api-access-24pk6\") pod \"multus-additional-cni-plugins-8f42z\" (UID: \"6207540f-5268-4df4-b0fa-94cd2e4e4a44\") " pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.250147 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.250095    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzxfp\" (UniqueName: \"kubernetes.io/projected/882218e0-217a-4d4f-aff1-121417a806e7-kube-api-access-mzxfp\") pod \"ovnkube-node-q6sq5\" (UID: \"882218e0-217a-4d4f-aff1-121417a806e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.252558 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.252512    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgdbl\" (UniqueName: \"kubernetes.io/projected/f2473042-4162-458d-8969-ae7e8668b0b3-kube-api-access-bgdbl\") pod \"node-resolver-qzw6h\" (UID: \"f2473042-4162-458d-8969-ae7e8668b0b3\") " pod="openshift-dns/node-resolver-qzw6h"
Jul 06 19:50:22.339586 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgnf2\" (UniqueName: \"kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2\") pod \"network-check-target-vqvgv\" (UID: \"1b321be0-8e73-4d70-91d7-e42975820cb4\") " pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:22.339809 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-system-cni-dir\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.339809 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-multus-socket-dir-parent\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.339809 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tftct\" (UniqueName: \"kubernetes.io/projected/f658f919-d15f-4648-949f-fce06f96211c-kube-api-access-tftct\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.339809 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339738    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-system-cni-dir\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.339809 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339746    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-multus-socket-dir-parent\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.339809 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-device-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-hostroot\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-kubelet-config\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339850    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-device-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4b68573-84ea-4f3d-980c-882767d83501-agent-certs\") pod \"konnectivity-agent-x9qff\" (UID: \"f4b68573-84ea-4f3d-980c-882767d83501\") " pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339883    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-hostroot\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-socket-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339909    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-kubelet-config\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339934    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-registration-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-etc-selinux\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-run-netns\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.339988    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-registration-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-var-lib-cni-bin\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340035    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-run-netns\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7m5z\" (UniqueName: \"kubernetes.io/projected/509ae1fb-ff23-4a78-acb5-5f55e5900408-kube-api-access-f7m5z\") pod \"node-ca-29qdv\" (UID: \"509ae1fb-ff23-4a78-acb5-5f55e5900408\") " pod="openshift-image-registry/node-ca-29qdv"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340075    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-var-lib-cni-bin\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.340153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340075    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-socket-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340101    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-etc-selinux\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340099    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce8dc260-f060-4662-9265-f8195b755570-cni-binary-copy\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340148    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340156    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-multus-cni-dir\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340206    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-multus-cni-dir\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-cnibin\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340276    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-multus-conf-dir\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340293    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-cnibin\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340345    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-etc-kubernetes\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-etc-kubernetes\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340357    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-multus-conf-dir\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/509ae1fb-ff23-4a78-acb5-5f55e5900408-host\") pod \"node-ca-29qdv\" (UID: \"509ae1fb-ff23-4a78-acb5-5f55e5900408\") " pod="openshift-image-registry/node-ca-29qdv"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340465    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/509ae1fb-ff23-4a78-acb5-5f55e5900408-host\") pod \"node-ca-29qdv\" (UID: \"509ae1fb-ff23-4a78-acb5-5f55e5900408\") " pod="openshift-image-registry/node-ca-29qdv"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340477    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-run-k8s-cni-cncf-io\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-sys-fs\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.340538    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340541    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-run-k8s-cni-cncf-io\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.341081 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340593    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-dbus\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340597    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f658f919-d15f-4648-949f-fce06f96211c-sys-fs\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.340609    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret podName:6feda960-34dc-47d7-bd85-4cbbc95bf5ba nodeName:}" failed. No retries permitted until 2026-07-06 19:50:22.84058621 +0000 UTC m=+3.152041436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret") pod "global-pull-secret-syncer-8x5db" (UID: "6feda960-34dc-47d7-bd85-4cbbc95bf5ba") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340538    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-dbus\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khcf5\" (UniqueName: \"kubernetes.io/projected/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-kube-api-access-khcf5\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340661    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce8dc260-f060-4662-9265-f8195b755570-cni-binary-copy\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-run-multus-certs\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340710    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-run-multus-certs\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340713    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-var-lib-kubelet\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce8dc260-f060-4662-9265-f8195b755570-multus-daemon-config\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340751    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-var-lib-kubelet\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4b68573-84ea-4f3d-980c-882767d83501-konnectivity-ca\") pod \"konnectivity-agent-x9qff\" (UID: \"f4b68573-84ea-4f3d-980c-882767d83501\") " pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/509ae1fb-ff23-4a78-acb5-5f55e5900408-serviceca\") pod \"node-ca-29qdv\" (UID: \"509ae1fb-ff23-4a78-acb5-5f55e5900408\") " pod="openshift-image-registry/node-ca-29qdv"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-var-lib-cni-multus\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340875    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-os-release\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.340899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqqkj\" (UniqueName: \"kubernetes.io/projected/ce8dc260-f060-4662-9265-f8195b755570-kube-api-access-fqqkj\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.342118 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.340922    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:22.343023 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.340974    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs podName:17b8e220-72ef-4627-9a4e-2e57eb63aa7c nodeName:}" failed. No retries permitted until 2026-07-06 19:50:22.840956002 +0000 UTC m=+3.152411222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs") pod "network-metrics-daemon-6dbm7" (UID: "17b8e220-72ef-4627-9a4e-2e57eb63aa7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:22.343023 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.341017    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-host-var-lib-cni-multus\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.343023 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.341083    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce8dc260-f060-4662-9265-f8195b755570-os-release\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.343023 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.341312    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/509ae1fb-ff23-4a78-acb5-5f55e5900408-serviceca\") pod \"node-ca-29qdv\" (UID: \"509ae1fb-ff23-4a78-acb5-5f55e5900408\") " pod="openshift-image-registry/node-ca-29qdv"
Jul 06 19:50:22.343023 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.341380    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce8dc260-f060-4662-9265-f8195b755570-multus-daemon-config\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.343023 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.341391    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4b68573-84ea-4f3d-980c-882767d83501-konnectivity-ca\") pod \"konnectivity-agent-x9qff\" (UID: \"f4b68573-84ea-4f3d-980c-882767d83501\") " pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:22.343023 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.342834    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4b68573-84ea-4f3d-980c-882767d83501-agent-certs\") pod \"konnectivity-agent-x9qff\" (UID: \"f4b68573-84ea-4f3d-980c-882767d83501\") " pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:22.353605 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.353525    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:50:22.353605 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.353559    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:50:22.353605 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.353577    2577 projected.go:196] Error preparing data for projected volume kube-api-access-xgnf2 for pod openshift-network-diagnostics/network-check-target-vqvgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:22.353872 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.353654    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2 podName:1b321be0-8e73-4d70-91d7-e42975820cb4 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:22.853632251 +0000 UTC m=+3.165087465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xgnf2" (UniqueName: "kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2") pod "network-check-target-vqvgv" (UID: "1b321be0-8e73-4d70-91d7-e42975820cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:22.355458 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.355425    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tftct\" (UniqueName: \"kubernetes.io/projected/f658f919-d15f-4648-949f-fce06f96211c-kube-api-access-tftct\") pod \"aws-ebs-csi-driver-node-6htxl\" (UID: \"f658f919-d15f-4648-949f-fce06f96211c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.355919 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.355897    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khcf5\" (UniqueName: \"kubernetes.io/projected/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-kube-api-access-khcf5\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:22.356076 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.356059    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqqkj\" (UniqueName: \"kubernetes.io/projected/ce8dc260-f060-4662-9265-f8195b755570-kube-api-access-fqqkj\") pod \"multus-mbg4d\" (UID: \"ce8dc260-f060-4662-9265-f8195b755570\") " pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.356172 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.356151    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7m5z\" (UniqueName: \"kubernetes.io/projected/509ae1fb-ff23-4a78-acb5-5f55e5900408-kube-api-access-f7m5z\") pod \"node-ca-29qdv\" (UID: \"509ae1fb-ff23-4a78-acb5-5f55e5900408\") " pod="openshift-image-registry/node-ca-29qdv"
Jul 06 19:50:22.417593 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.417555    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:50:22.427679 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.427645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8f42z"
Jul 06 19:50:22.437817 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.437791    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:22.449966 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.449939    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mms6t"
Jul 06 19:50:22.456979 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.456956    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hbbkv"
Jul 06 19:50:22.463945 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.463917    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qzw6h"
Jul 06 19:50:22.472957 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.472927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mbg4d"
Jul 06 19:50:22.492021 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.491984    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl"
Jul 06 19:50:22.498226 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.498198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:22.500361 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.500334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-29qdv"
Jul 06 19:50:22.646854 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.646822    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:50:22.844289 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.844251    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:22.844483 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.844349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:22.844483 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.844422    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:22.844483 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.844450    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:22.844666 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.844495    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs podName:17b8e220-72ef-4627-9a4e-2e57eb63aa7c nodeName:}" failed. No retries permitted until 2026-07-06 19:50:23.844471158 +0000 UTC m=+4.155926363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs") pod "network-metrics-daemon-6dbm7" (UID: "17b8e220-72ef-4627-9a4e-2e57eb63aa7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:22.844666 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.844531    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret podName:6feda960-34dc-47d7-bd85-4cbbc95bf5ba nodeName:}" failed. No retries permitted until 2026-07-06 19:50:23.844518779 +0000 UTC m=+4.155973983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret") pod "global-pull-secret-syncer-8x5db" (UID: "6feda960-34dc-47d7-bd85-4cbbc95bf5ba") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:22.945517 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:22.945423    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgnf2\" (UniqueName: \"kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2\") pod \"network-check-target-vqvgv\" (UID: \"1b321be0-8e73-4d70-91d7-e42975820cb4\") " pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:22.945699 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.945580    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:50:22.945699 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.945601    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:50:22.945699 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.945613    2577 projected.go:196] Error preparing data for projected volume kube-api-access-xgnf2 for pod openshift-network-diagnostics/network-check-target-vqvgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:22.945866 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:22.945692    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2 podName:1b321be0-8e73-4d70-91d7-e42975820cb4 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:23.945667574 +0000 UTC m=+4.257122789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgnf2" (UniqueName: "kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2") pod "network-check-target-vqvgv" (UID: "1b321be0-8e73-4d70-91d7-e42975820cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:23.017416 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:23.017387    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 19:50:23.173418 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:23.173372    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:45:22 +0000 UTC" deadline="2028-04-18 06:31:18.358905612 +0000 UTC"
Jul 06 19:50:23.173418 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:23.173411    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15634h40m55.18550007s"
Jul 06 19:50:23.332790 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:23.332697    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:23.332790 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:23.332748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:23.333010 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:23.332858    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:23.333071 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:23.333017    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:23.434403 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:23.434361    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882218e0_217a_4d4f_aff1_121417a806e7.slice/crio-359b17fbdc687dc40811f483b4e0f6014428476c7eb6f8999b413258f4c2a0e2 WatchSource:0}: Error finding container 359b17fbdc687dc40811f483b4e0f6014428476c7eb6f8999b413258f4c2a0e2: Status 404 returned error can't find the container with id 359b17fbdc687dc40811f483b4e0f6014428476c7eb6f8999b413258f4c2a0e2
Jul 06 19:50:23.436503 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:23.436468    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce8dc260_f060_4662_9265_f8195b755570.slice/crio-b2b85cd30a349f6c25e55bca954d7b781cffd5b242b4697596dc5638ca9661da WatchSource:0}: Error finding container b2b85cd30a349f6c25e55bca954d7b781cffd5b242b4697596dc5638ca9661da: Status 404 returned error can't find the container with id b2b85cd30a349f6c25e55bca954d7b781cffd5b242b4697596dc5638ca9661da
Jul 06 19:50:23.440485 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:23.440456    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod509ae1fb_ff23_4a78_acb5_5f55e5900408.slice/crio-467bce31b61f683a0f572781fc86c317e8f85c5b3537aa27d47a0e5f55efa4eb WatchSource:0}: Error finding container 467bce31b61f683a0f572781fc86c317e8f85c5b3537aa27d47a0e5f55efa4eb: Status 404 returned error can't find the container with id 467bce31b61f683a0f572781fc86c317e8f85c5b3537aa27d47a0e5f55efa4eb
Jul 06 19:50:23.442028 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:23.441905    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2473042_4162_458d_8969_ae7e8668b0b3.slice/crio-4015472ef24d943c0809b3a2be9feb22fda5f73c4c638b2d61e5794bc789c7ad WatchSource:0}: Error finding container 4015472ef24d943c0809b3a2be9feb22fda5f73c4c638b2d61e5794bc789c7ad: Status 404 returned error can't find the container with id 4015472ef24d943c0809b3a2be9feb22fda5f73c4c638b2d61e5794bc789c7ad
Jul 06 19:50:23.443072 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:23.443048    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189aff81_6dc1_4c79_b9c8_d6807cd546fb.slice/crio-4dcee0c2e941797a49df4928f15b6de471e519d9d8790d48f99d29b84d7ddbe3 WatchSource:0}: Error finding container 4dcee0c2e941797a49df4928f15b6de471e519d9d8790d48f99d29b84d7ddbe3: Status 404 returned error can't find the container with id 4dcee0c2e941797a49df4928f15b6de471e519d9d8790d48f99d29b84d7ddbe3
Jul 06 19:50:23.466610 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:23.466579    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf658f919_d15f_4648_949f_fce06f96211c.slice/crio-f5fa642968903bc113b55d02bb5b6c2fd3d135c4cd39949208e191b446ce7e71 WatchSource:0}: Error finding container f5fa642968903bc113b55d02bb5b6c2fd3d135c4cd39949208e191b446ce7e71: Status 404 returned error can't find the container with id f5fa642968903bc113b55d02bb5b6c2fd3d135c4cd39949208e191b446ce7e71
Jul 06 19:50:23.853357 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:23.853104    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:23.853542 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:23.853291    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:23.853542 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:23.853427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:23.853542 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:23.853514    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs podName:17b8e220-72ef-4627-9a4e-2e57eb63aa7c nodeName:}" failed. No retries permitted until 2026-07-06 19:50:25.853486355 +0000 UTC m=+6.164941561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs") pod "network-metrics-daemon-6dbm7" (UID: "17b8e220-72ef-4627-9a4e-2e57eb63aa7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:23.853686 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:23.853551    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:23.853686 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:23.853601    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret podName:6feda960-34dc-47d7-bd85-4cbbc95bf5ba nodeName:}" failed. No retries permitted until 2026-07-06 19:50:25.853588187 +0000 UTC m=+6.165043393 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret") pod "global-pull-secret-syncer-8x5db" (UID: "6feda960-34dc-47d7-bd85-4cbbc95bf5ba") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:23.956385 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:23.953984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgnf2\" (UniqueName: \"kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2\") pod \"network-check-target-vqvgv\" (UID: \"1b321be0-8e73-4d70-91d7-e42975820cb4\") " pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:23.956385 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:23.954227    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:50:23.956385 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:23.954258    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:50:23.956385 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:23.954302    2577 projected.go:196] Error preparing data for projected volume kube-api-access-xgnf2 for pod openshift-network-diagnostics/network-check-target-vqvgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:23.956385 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:23.954412    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2 podName:1b321be0-8e73-4d70-91d7-e42975820cb4 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:25.954390919 +0000 UTC m=+6.265846138 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgnf2" (UniqueName: "kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2") pod "network-check-target-vqvgv" (UID: "1b321be0-8e73-4d70-91d7-e42975820cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:24.173964 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.173897    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:45:22 +0000 UTC" deadline="2028-04-06 16:37:00.99135137 +0000 UTC"
Jul 06 19:50:24.173964 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.173961    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15356h46m36.817394604s"
Jul 06 19:50:24.340330 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.339748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:24.340330 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:24.339864    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:24.354912 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.354834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-90.ec2.internal" event={"ID":"36e6979a7a16f66ce876f6ff5066432c","Type":"ContainerStarted","Data":"462bcb3e3eb3745c46daffec400fb2cce21e204b2c06074a62ee4ee5fe2c4109"}
Jul 06 19:50:24.371867 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.371334    2577 generic.go:356] "Generic (PLEG): container finished" podID="4701a56854965b6d68e38c039cde7250" containerID="510d09eaea68358003c75a16b34fd7e698be3c4a6338d69cc727b80e26db89dc" exitCode=0
Jul 06 19:50:24.371867 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.371420    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal" event={"ID":"4701a56854965b6d68e38c039cde7250","Type":"ContainerDied","Data":"510d09eaea68358003c75a16b34fd7e698be3c4a6338d69cc727b80e26db89dc"}
Jul 06 19:50:24.373713 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.373459    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-90.ec2.internal" podStartSLOduration=3.373440157 podStartE2EDuration="3.373440157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:24.373188428 +0000 UTC m=+4.684643647" watchObservedRunningTime="2026-07-06 19:50:24.373440157 +0000 UTC m=+4.684895379"
Jul 06 19:50:24.379244 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.379212    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x9qff" event={"ID":"f4b68573-84ea-4f3d-980c-882767d83501","Type":"ContainerStarted","Data":"3cf3f3b66e180a5288200402c2e7853039f3ba02027252aec9fa6b6280ec4825"}
Jul 06 19:50:24.389957 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.389703    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mms6t" event={"ID":"47dc2bc5-2e8c-47a7-9a1e-5be37758f974","Type":"ContainerStarted","Data":"142724c3b63b2f15cfb26bae79ee0656a373afaf0d41248a765af37b26e09236"}
Jul 06 19:50:24.398250 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.398190    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl" event={"ID":"f658f919-d15f-4648-949f-fce06f96211c","Type":"ContainerStarted","Data":"f5fa642968903bc113b55d02bb5b6c2fd3d135c4cd39949208e191b446ce7e71"}
Jul 06 19:50:24.410364 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.407962    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-29qdv" event={"ID":"509ae1fb-ff23-4a78-acb5-5f55e5900408","Type":"ContainerStarted","Data":"467bce31b61f683a0f572781fc86c317e8f85c5b3537aa27d47a0e5f55efa4eb"}
Jul 06 19:50:24.410865 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.410560    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mbg4d" event={"ID":"ce8dc260-f060-4662-9265-f8195b755570","Type":"ContainerStarted","Data":"b2b85cd30a349f6c25e55bca954d7b781cffd5b242b4697596dc5638ca9661da"}
Jul 06 19:50:24.422458 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.422349    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5" event={"ID":"882218e0-217a-4d4f-aff1-121417a806e7","Type":"ContainerStarted","Data":"359b17fbdc687dc40811f483b4e0f6014428476c7eb6f8999b413258f4c2a0e2"}
Jul 06 19:50:24.433164 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.433119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f42z" event={"ID":"6207540f-5268-4df4-b0fa-94cd2e4e4a44","Type":"ContainerStarted","Data":"58079c3085157a626be11866f12f62f49c22c6e2e7c623ef6c1a5b5d10a5b179"}
Jul 06 19:50:24.458353 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.458317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hbbkv" event={"ID":"189aff81-6dc1-4c79-b9c8-d6807cd546fb","Type":"ContainerStarted","Data":"4dcee0c2e941797a49df4928f15b6de471e519d9d8790d48f99d29b84d7ddbe3"}
Jul 06 19:50:24.470965 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:24.470925    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qzw6h" event={"ID":"f2473042-4162-458d-8969-ae7e8668b0b3","Type":"ContainerStarted","Data":"4015472ef24d943c0809b3a2be9feb22fda5f73c4c638b2d61e5794bc789c7ad"}
Jul 06 19:50:25.331846 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:25.331812    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:25.332414 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:25.331946    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:25.332414 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:25.332363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:25.332522 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:25.332467    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:25.493493 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:25.493284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal" event={"ID":"4701a56854965b6d68e38c039cde7250","Type":"ContainerStarted","Data":"4dcbf066189789b7a60f90dbc0963c7752c7d6a38bcbabc0a3c040b940b7128d"}
Jul 06 19:50:25.871737 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:25.871698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:25.871938 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:25.871763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:25.880375 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:25.880341    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:25.880537 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:25.880441    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret podName:6feda960-34dc-47d7-bd85-4cbbc95bf5ba nodeName:}" failed. No retries permitted until 2026-07-06 19:50:29.880419177 +0000 UTC m=+10.191874378 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret") pod "global-pull-secret-syncer-8x5db" (UID: "6feda960-34dc-47d7-bd85-4cbbc95bf5ba") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:25.880946 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:25.880928    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:25.881014 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:25.880986    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs podName:17b8e220-72ef-4627-9a4e-2e57eb63aa7c nodeName:}" failed. No retries permitted until 2026-07-06 19:50:29.880964342 +0000 UTC m=+10.192419546 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs") pod "network-metrics-daemon-6dbm7" (UID: "17b8e220-72ef-4627-9a4e-2e57eb63aa7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:25.972597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:25.972554    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgnf2\" (UniqueName: \"kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2\") pod \"network-check-target-vqvgv\" (UID: \"1b321be0-8e73-4d70-91d7-e42975820cb4\") " pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:25.972786 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:25.972729    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:50:25.972786 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:25.972746    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:50:25.972786 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:25.972759    2577 projected.go:196] Error preparing data for projected volume kube-api-access-xgnf2 for pod openshift-network-diagnostics/network-check-target-vqvgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:25.972945 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:25.972818    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2 podName:1b321be0-8e73-4d70-91d7-e42975820cb4 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:29.972799854 +0000 UTC m=+10.284255072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgnf2" (UniqueName: "kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2") pod "network-check-target-vqvgv" (UID: "1b321be0-8e73-4d70-91d7-e42975820cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:26.335007 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:26.334929    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:26.335442 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:26.335049    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:27.331843 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:27.331755    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:27.331843 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:27.331793    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:27.332083 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:27.331895    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:27.332366 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:27.332318    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:28.332773 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:28.332464    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:28.332773 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:28.332597    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:29.332840 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:29.332805    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:29.333444 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:29.332822    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:29.333444 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:29.332969    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:29.333444 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:29.333152    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:29.909043 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:29.909004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:29.909239 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:29.909062    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:29.909239 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:29.909193    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:29.909372 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:29.909193    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:29.909372 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:29.909267    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret podName:6feda960-34dc-47d7-bd85-4cbbc95bf5ba nodeName:}" failed. No retries permitted until 2026-07-06 19:50:37.909246113 +0000 UTC m=+18.220701316 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret") pod "global-pull-secret-syncer-8x5db" (UID: "6feda960-34dc-47d7-bd85-4cbbc95bf5ba") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:29.909372 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:29.909298    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs podName:17b8e220-72ef-4627-9a4e-2e57eb63aa7c nodeName:}" failed. No retries permitted until 2026-07-06 19:50:37.9092851 +0000 UTC m=+18.220740310 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs") pod "network-metrics-daemon-6dbm7" (UID: "17b8e220-72ef-4627-9a4e-2e57eb63aa7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:30.010037 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:30.010005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgnf2\" (UniqueName: \"kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2\") pod \"network-check-target-vqvgv\" (UID: \"1b321be0-8e73-4d70-91d7-e42975820cb4\") " pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:30.010248 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:30.010198    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:50:30.010248 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:30.010220    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:50:30.010248 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:30.010232    2577 projected.go:196] Error preparing data for projected volume kube-api-access-xgnf2 for pod openshift-network-diagnostics/network-check-target-vqvgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:30.010421 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:30.010294    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2 podName:1b321be0-8e73-4d70-91d7-e42975820cb4 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:38.010273979 +0000 UTC m=+18.321729178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgnf2" (UniqueName: "kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2") pod "network-check-target-vqvgv" (UID: "1b321be0-8e73-4d70-91d7-e42975820cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:30.333636 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:30.333558    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:30.334201 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:30.333696    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:31.332147 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:31.332113    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:31.332347 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:31.332268    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:31.332347 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:31.332277    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:31.332447 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:31.332373    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:32.332706 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:32.332662    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:32.333167 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:32.332778    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:33.332389 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:33.332350    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:33.332571 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:33.332368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:33.332571 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:33.332508    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:33.332663 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:33.332599    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:34.332430 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:34.332393    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:34.332877 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:34.332524    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:35.332259 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:35.332218    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:35.332447 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:35.332218    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:35.332447 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:35.332348    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:35.332447 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:35.332440    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:36.332380 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:36.332351    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:36.332610 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:36.332576    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:37.331871 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:37.331834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:37.332086 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:37.331834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:37.332086 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:37.331972    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:37.332086 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:37.332050    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:37.969647 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:37.969617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:37.969647 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:37.969658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:37.970086 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:37.969764    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:37.970086 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:37.969765    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:37.970086 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:37.969825    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs podName:17b8e220-72ef-4627-9a4e-2e57eb63aa7c nodeName:}" failed. No retries permitted until 2026-07-06 19:50:53.969807791 +0000 UTC m=+34.281263009 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs") pod "network-metrics-daemon-6dbm7" (UID: "17b8e220-72ef-4627-9a4e-2e57eb63aa7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:37.970086 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:37.969839    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret podName:6feda960-34dc-47d7-bd85-4cbbc95bf5ba nodeName:}" failed. No retries permitted until 2026-07-06 19:50:53.969833391 +0000 UTC m=+34.281288589 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret") pod "global-pull-secret-syncer-8x5db" (UID: "6feda960-34dc-47d7-bd85-4cbbc95bf5ba") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:38.070756 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:38.070725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgnf2\" (UniqueName: \"kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2\") pod \"network-check-target-vqvgv\" (UID: \"1b321be0-8e73-4d70-91d7-e42975820cb4\") " pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:38.070939 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:38.070904    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:50:38.070939 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:38.070926    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:50:38.070939 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:38.070937    2577 projected.go:196] Error preparing data for projected volume kube-api-access-xgnf2 for pod openshift-network-diagnostics/network-check-target-vqvgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:38.071086 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:38.071001    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2 podName:1b321be0-8e73-4d70-91d7-e42975820cb4 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:54.070983154 +0000 UTC m=+34.382438356 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgnf2" (UniqueName: "kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2") pod "network-check-target-vqvgv" (UID: "1b321be0-8e73-4d70-91d7-e42975820cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:38.332047 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:38.331956    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:38.332234 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:38.332093    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:39.331922 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:39.331887    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:39.332396 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:39.331887    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:39.332396 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:39.332017    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:39.332396 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:39.332095    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:40.332390 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:40.332354    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:40.332787 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:40.332444    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:41.332240 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:41.332199    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:41.332408 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:41.332199    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:41.332408 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:41.332319    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:41.332408 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:41.332391    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:42.332777 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.332395    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:42.332777 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:42.332718    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:42.529328 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.529167    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x9qff" event={"ID":"f4b68573-84ea-4f3d-980c-882767d83501","Type":"ContainerStarted","Data":"c0f0df77919c773a27f2850f7ab59a6dc4bdb6d3a808f6dce3cc3ff1c973b4e9"}
Jul 06 19:50:42.530509 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.530489    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl" event={"ID":"f658f919-d15f-4648-949f-fce06f96211c","Type":"ContainerStarted","Data":"453bc1468c7c9561e91c6a4f600a3c9f9de151bdb8f2f890ddbd3a98db60040e"}
Jul 06 19:50:42.531721 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.531701    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-29qdv" event={"ID":"509ae1fb-ff23-4a78-acb5-5f55e5900408","Type":"ContainerStarted","Data":"8251be7f264c7bcf8e8b050c165dd182f78ae37d31fa055bdff9b578062dde54"}
Jul 06 19:50:42.533023 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.533005    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mbg4d" event={"ID":"ce8dc260-f060-4662-9265-f8195b755570","Type":"ContainerStarted","Data":"5c7a407cea197e94c3dc4f1f5268805d27112209d913acf992384c8f6be6cd55"}
Jul 06 19:50:42.534606 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.534590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5" event={"ID":"882218e0-217a-4d4f-aff1-121417a806e7","Type":"ContainerStarted","Data":"be7c7ad59ebca9fc17266c52c59c0c7726d1a9c21ac97f7c2a7cec6f8661d8d1"}
Jul 06 19:50:42.534685 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.534640    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5" event={"ID":"882218e0-217a-4d4f-aff1-121417a806e7","Type":"ContainerStarted","Data":"212c71e06f929171da11a5d00d344f1ee3da08f2b5a5a7c4e87f86473b1899df"}
Jul 06 19:50:42.535942 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.535920    2577 generic.go:356] "Generic (PLEG): container finished" podID="6207540f-5268-4df4-b0fa-94cd2e4e4a44" containerID="6df252afbe7395ee8d9167606b6043ed59711ae8029d609ccb58a039f3040c4c" exitCode=0
Jul 06 19:50:42.536025 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.535986    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f42z" event={"ID":"6207540f-5268-4df4-b0fa-94cd2e4e4a44","Type":"ContainerDied","Data":"6df252afbe7395ee8d9167606b6043ed59711ae8029d609ccb58a039f3040c4c"}
Jul 06 19:50:42.537478 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.537460    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hbbkv" event={"ID":"189aff81-6dc1-4c79-b9c8-d6807cd546fb","Type":"ContainerStarted","Data":"4fdd01ceff9ee8844e7dc2affd2c46cebd728e24323ea8f2437ca79aee9946cb"}
Jul 06 19:50:42.538804 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.538770    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qzw6h" event={"ID":"f2473042-4162-458d-8969-ae7e8668b0b3","Type":"ContainerStarted","Data":"fef4cca048a20603efac25ab86e774348b431fb142d607da5a2f532959977bbd"}
Jul 06 19:50:42.552138 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.552091    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-90.ec2.internal" podStartSLOduration=21.552056167 podStartE2EDuration="21.552056167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:25.509661253 +0000 UTC m=+5.821116528" watchObservedRunningTime="2026-07-06 19:50:42.552056167 +0000 UTC m=+22.863511390"
Jul 06 19:50:42.553043 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.553001    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x9qff" podStartSLOduration=8.546194475 podStartE2EDuration="22.552985657s" totalImagesPullingTime="14.006791182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:42.551983009 +0000 UTC m=+22.863438229" watchObservedRunningTime="2026-07-06 19:50:42.552985657 +0000 UTC m=+22.864440879"
Jul 06 19:50:42.573053 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.573019    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qzw6h" podStartSLOduration=3.927288504 podStartE2EDuration="22.573008664s" totalImagesPullingTime="18.64572016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:42.572709873 +0000 UTC m=+22.884165095" watchObservedRunningTime="2026-07-06 19:50:42.573008664 +0000 UTC m=+22.884463884"
Jul 06 19:50:42.588600 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.588540    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-29qdv" podStartSLOduration=3.935755527 podStartE2EDuration="22.588528223s" totalImagesPullingTime="18.652772696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:42.588048637 +0000 UTC m=+22.899503859" watchObservedRunningTime="2026-07-06 19:50:42.588528223 +0000 UTC m=+22.899983443"
Jul 06 19:50:42.627073 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:42.627023    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hbbkv" podStartSLOduration=3.958238124 podStartE2EDuration="22.62701029s" totalImagesPullingTime="18.668772166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:42.626496036 +0000 UTC m=+22.937951254" watchObservedRunningTime="2026-07-06 19:50:42.62701029 +0000 UTC m=+22.938465511"
Jul 06 19:50:43.332245 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:43.332210    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:43.332453 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:43.332251    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:43.332453 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:43.332309    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:43.332453 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:43.332441    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:43.542547 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:43.542515    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mms6t" event={"ID":"47dc2bc5-2e8c-47a7-9a1e-5be37758f974","Type":"ContainerStarted","Data":"6bae43d5e38042aa416d965b922f3d8c815fcb044fadd08ac6081cb8ba144b1c"}
Jul 06 19:50:43.546065 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:43.546029    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5" event={"ID":"882218e0-217a-4d4f-aff1-121417a806e7","Type":"ContainerStarted","Data":"2ac44c27c6ce6b21010f79b4f64f2edfc30a7a5519bcf2833a36f18412c0751c"}
Jul 06 19:50:43.546199 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:43.546114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5" event={"ID":"882218e0-217a-4d4f-aff1-121417a806e7","Type":"ContainerStarted","Data":"e279a0d87a196057ddb8de0e30a15e3478b48c502a3e170fc00c69fed9735579"}
Jul 06 19:50:43.546199 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:43.546164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5" event={"ID":"882218e0-217a-4d4f-aff1-121417a806e7","Type":"ContainerStarted","Data":"a0fd5fbdba420d802de6ccc776fd5ea72c56482bfcf9b8f12a0f431152b71ba6"}
Jul 06 19:50:43.546346 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:43.546229    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5" event={"ID":"882218e0-217a-4d4f-aff1-121417a806e7","Type":"ContainerStarted","Data":"a3f74186edf0027831afef1fcb488a3ca233bbae0866d1ad654b068214ac3b68"}
Jul 06 19:50:43.557957 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:43.557913    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mbg4d" podStartSLOduration=4.85076804 podStartE2EDuration="23.557899692s" totalImagesPullingTime="18.707131652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:42.645084728 +0000 UTC m=+22.956539945" watchObservedRunningTime="2026-07-06 19:50:43.557899692 +0000 UTC m=+23.869354913"
Jul 06 19:50:43.558225 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:43.558200    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mms6t" podStartSLOduration=4.896924895 podStartE2EDuration="23.558193761s" totalImagesPullingTime="18.661268866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:43.557708246 +0000 UTC m=+23.869163469" watchObservedRunningTime="2026-07-06 19:50:43.558193761 +0000 UTC m=+23.869648982"
Jul 06 19:50:43.729905 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:43.729874    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 19:50:44.189514 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:44.189404    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T19:50:43.729901525Z","UUID":"b2181426-46e4-4375-9dbb-fbee1fead316","Handler":null,"Name":"","Endpoint":""}
Jul 06 19:50:44.190987 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:44.190970    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 19:50:44.191112 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:44.191008    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 19:50:44.332399 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:44.332368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:44.332574 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:44.332486    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:44.549920 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:44.549845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl" event={"ID":"f658f919-d15f-4648-949f-fce06f96211c","Type":"ContainerStarted","Data":"36924d8302392afbb775432acf38cdbd94a15494c0a692d56bc0c36d4eb82c23"}
Jul 06 19:50:45.332758 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:45.332584    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:45.333085 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:45.332655    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:45.333085 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:45.332844    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:45.333085 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:45.332896    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:45.553528 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:45.553494    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl" event={"ID":"f658f919-d15f-4648-949f-fce06f96211c","Type":"ContainerStarted","Data":"824f82ffc3cdf689c3b07ae9bbd363dd3a32442f05c1a862627f29fe53d5ff32"}
Jul 06 19:50:45.556642 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:45.556612    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5" event={"ID":"882218e0-217a-4d4f-aff1-121417a806e7","Type":"ContainerStarted","Data":"86f3e64c53b2b18c46293719f6eb370f8901bafc4d532a68c3d51eee5b7d5473"}
Jul 06 19:50:45.573885 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:45.573832    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6htxl" podStartSLOduration=4.711017736 podStartE2EDuration="25.57381492s" totalImagesPullingTime="20.862797184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:45.573747203 +0000 UTC m=+25.885202426" watchObservedRunningTime="2026-07-06 19:50:45.57381492 +0000 UTC m=+25.885270141"
Jul 06 19:50:46.332621 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:46.332585    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:46.332845 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:46.332722    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:47.155862 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:47.155826    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:47.156583 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:47.156563    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:47.332496 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:47.332420    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:47.332649 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:47.332420    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:47.332649 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:47.332532    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:47.332649 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:47.332626    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:47.561632 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:47.561599    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:47.562227 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:47.562204    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x9qff"
Jul 06 19:50:48.332192 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:48.331988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:48.332705 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:48.332287    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:48.567114 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:48.567077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5" event={"ID":"882218e0-217a-4d4f-aff1-121417a806e7","Type":"ContainerStarted","Data":"7c751abba3df1d64c39806e1d0f4af68ad13f1dba1a4753f656d5d2a60d73ab9"}
Jul 06 19:50:48.567453 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:48.567424    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:48.567704 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:48.567497    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:48.568916 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:48.568890    2577 generic.go:356] "Generic (PLEG): container finished" podID="6207540f-5268-4df4-b0fa-94cd2e4e4a44" containerID="0611fa0fac02c1f0f5650d88daefb5f9539c82dc6fdf99152d1e45a6d34affd6" exitCode=0
Jul 06 19:50:48.569007 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:48.568967    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f42z" event={"ID":"6207540f-5268-4df4-b0fa-94cd2e4e4a44","Type":"ContainerDied","Data":"0611fa0fac02c1f0f5650d88daefb5f9539c82dc6fdf99152d1e45a6d34affd6"}
Jul 06 19:50:48.583369 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:48.583351    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:48.598830 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:48.598792    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5" podStartSLOduration=9.812331537 podStartE2EDuration="28.59878089s" totalImagesPullingTime="18.786449353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:48.598637135 +0000 UTC m=+28.910092360" watchObservedRunningTime="2026-07-06 19:50:48.59878089 +0000 UTC m=+28.910236111"
Jul 06 19:50:49.332377 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:49.332348    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:49.332377 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:49.332362    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:49.333018 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:49.332487    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:49.333018 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:49.332600    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:49.572552 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:49.572507    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:49.589205 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:49.589123    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:50:49.832170 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:49.831994    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6dbm7"]
Jul 06 19:50:49.832333 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:49.832247    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:49.832375 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:49.832328    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:49.835485 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:49.835462    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vqvgv"]
Jul 06 19:50:49.835601 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:49.835550    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:49.835671 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:49.835625    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:49.835994 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:49.835973    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8x5db"]
Jul 06 19:50:49.836065 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:49.836045    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:49.836146 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:49.836114    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:50.576594 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:50.576558    2577 generic.go:356] "Generic (PLEG): container finished" podID="6207540f-5268-4df4-b0fa-94cd2e4e4a44" containerID="4b8dafb420d4c8c4a69cd65260d4f8923108d4ecaafc140b88934230b597ec3d" exitCode=0
Jul 06 19:50:50.576985 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:50.576639    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f42z" event={"ID":"6207540f-5268-4df4-b0fa-94cd2e4e4a44","Type":"ContainerDied","Data":"4b8dafb420d4c8c4a69cd65260d4f8923108d4ecaafc140b88934230b597ec3d"}
Jul 06 19:50:51.331976 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:51.331941    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:51.331976 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:51.331977    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:51.332198 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:51.331984    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:51.332198 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:51.332062    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:51.332279 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:51.332242    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:51.332370 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:51.332349    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:52.583754 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:52.583717    2577 generic.go:356] "Generic (PLEG): container finished" podID="6207540f-5268-4df4-b0fa-94cd2e4e4a44" containerID="07d5bb297b3c5646ac9964c619aef01aed428e67f10bf9a1ceb1ba81e8c0b058" exitCode=0
Jul 06 19:50:52.584298 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:52.583762    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f42z" event={"ID":"6207540f-5268-4df4-b0fa-94cd2e4e4a44","Type":"ContainerDied","Data":"07d5bb297b3c5646ac9964c619aef01aed428e67f10bf9a1ceb1ba81e8c0b058"}
Jul 06 19:50:53.332802 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:53.332768    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:53.332995 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:53.332883    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:53.332995 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:53.332899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:53.332995 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:53.332892    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqvgv" podUID="1b321be0-8e73-4d70-91d7-e42975820cb4"
Jul 06 19:50:53.332995 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:53.332974    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8x5db" podUID="6feda960-34dc-47d7-bd85-4cbbc95bf5ba"
Jul 06 19:50:53.333221 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:53.333076    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:50:53.991103 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:53.991070    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:53.991547 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:53.991126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:53.991547 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:53.991247    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:53.991547 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:53.991283    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:53.991547 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:53.991316    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret podName:6feda960-34dc-47d7-bd85-4cbbc95bf5ba nodeName:}" failed. No retries permitted until 2026-07-06 19:51:25.991294473 +0000 UTC m=+66.302749675 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret") pod "global-pull-secret-syncer-8x5db" (UID: "6feda960-34dc-47d7-bd85-4cbbc95bf5ba") : object "kube-system"/"original-pull-secret" not registered
Jul 06 19:50:53.991547 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:53.991336    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs podName:17b8e220-72ef-4627-9a4e-2e57eb63aa7c nodeName:}" failed. No retries permitted until 2026-07-06 19:51:25.991324757 +0000 UTC m=+66.302779959 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs") pod "network-metrics-daemon-6dbm7" (UID: "17b8e220-72ef-4627-9a4e-2e57eb63aa7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 19:50:54.092483 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.092449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgnf2\" (UniqueName: \"kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2\") pod \"network-check-target-vqvgv\" (UID: \"1b321be0-8e73-4d70-91d7-e42975820cb4\") " pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:54.092667 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:54.092634    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 19:50:54.092667 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:54.092664    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 19:50:54.092744 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:54.092675    2577 projected.go:196] Error preparing data for projected volume kube-api-access-xgnf2 for pod openshift-network-diagnostics/network-check-target-vqvgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:54.092744 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:54.092723    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2 podName:1b321be0-8e73-4d70-91d7-e42975820cb4 nodeName:}" failed. No retries permitted until 2026-07-06 19:51:26.092709471 +0000 UTC m=+66.404164670 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgnf2" (UniqueName: "kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2") pod "network-check-target-vqvgv" (UID: "1b321be0-8e73-4d70-91d7-e42975820cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 19:50:54.798603 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.798576    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-90.ec2.internal" event="NodeReady"
Jul 06 19:50:54.798783 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.798721    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 19:50:54.852788 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.852748    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rbkts"]
Jul 06 19:50:54.855024 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.855001    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lqm67"]
Jul 06 19:50:54.855149 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.855133    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rbkts"
Jul 06 19:50:54.857305 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.857257    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:54.857958 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.857936    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xpgq7\""
Jul 06 19:50:54.858067 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.857982    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 19:50:54.858067 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.857936    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 19:50:54.858067 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.857942    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 19:50:54.860390 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.860331    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 19:50:54.860481 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.860468    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 19:50:54.863040 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.861455    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-45vdg\""
Jul 06 19:50:54.866544 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.866412    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lqm67"]
Jul 06 19:50:54.867676 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.867653    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rbkts"]
Jul 06 19:50:54.998902 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.998864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/35a35874-da0b-41fe-b1d6-25bd8082e927-config-volume\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:54.998902 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.998899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:54.999393 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.998940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwmnc\" (UniqueName: \"kubernetes.io/projected/e0a787a4-c59e-48e3-b0ac-736de6af9b84-kube-api-access-mwmnc\") pod \"ingress-canary-rbkts\" (UID: \"e0a787a4-c59e-48e3-b0ac-736de6af9b84\") " pod="openshift-ingress-canary/ingress-canary-rbkts"
Jul 06 19:50:54.999393 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.998962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35a35874-da0b-41fe-b1d6-25bd8082e927-tmp-dir\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:54.999393 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.998977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2t9q\" (UniqueName: \"kubernetes.io/projected/35a35874-da0b-41fe-b1d6-25bd8082e927-kube-api-access-h2t9q\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:54.999393 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:54.999045    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0a787a4-c59e-48e3-b0ac-736de6af9b84-cert\") pod \"ingress-canary-rbkts\" (UID: \"e0a787a4-c59e-48e3-b0ac-736de6af9b84\") " pod="openshift-ingress-canary/ingress-canary-rbkts"
Jul 06 19:50:55.099793 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.099748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwmnc\" (UniqueName: \"kubernetes.io/projected/e0a787a4-c59e-48e3-b0ac-736de6af9b84-kube-api-access-mwmnc\") pod \"ingress-canary-rbkts\" (UID: \"e0a787a4-c59e-48e3-b0ac-736de6af9b84\") " pod="openshift-ingress-canary/ingress-canary-rbkts"
Jul 06 19:50:55.099984 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.099812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35a35874-da0b-41fe-b1d6-25bd8082e927-tmp-dir\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:55.099984 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.099838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2t9q\" (UniqueName: \"kubernetes.io/projected/35a35874-da0b-41fe-b1d6-25bd8082e927-kube-api-access-h2t9q\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:55.099984 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.099873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0a787a4-c59e-48e3-b0ac-736de6af9b84-cert\") pod \"ingress-canary-rbkts\" (UID: \"e0a787a4-c59e-48e3-b0ac-736de6af9b84\") " pod="openshift-ingress-canary/ingress-canary-rbkts"
Jul 06 19:50:55.100161 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.099998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/35a35874-da0b-41fe-b1d6-25bd8082e927-config-volume\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:55.100161 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.100031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:55.100290 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:55.100191    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:50:55.100290 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:55.100259    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls podName:35a35874-da0b-41fe-b1d6-25bd8082e927 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:55.600240567 +0000 UTC m=+35.911695769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls") pod "dns-default-lqm67" (UID: "35a35874-da0b-41fe-b1d6-25bd8082e927") : secret "dns-default-metrics-tls" not found
Jul 06 19:50:55.100290 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.100257    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35a35874-da0b-41fe-b1d6-25bd8082e927-tmp-dir\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:55.100607 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.100586    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/35a35874-da0b-41fe-b1d6-25bd8082e927-config-volume\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:55.104453 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.104430    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0a787a4-c59e-48e3-b0ac-736de6af9b84-cert\") pod \"ingress-canary-rbkts\" (UID: \"e0a787a4-c59e-48e3-b0ac-736de6af9b84\") " pod="openshift-ingress-canary/ingress-canary-rbkts"
Jul 06 19:50:55.108945 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.108900    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2t9q\" (UniqueName: \"kubernetes.io/projected/35a35874-da0b-41fe-b1d6-25bd8082e927-kube-api-access-h2t9q\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:55.109067 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.109046    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwmnc\" (UniqueName: \"kubernetes.io/projected/e0a787a4-c59e-48e3-b0ac-736de6af9b84-kube-api-access-mwmnc\") pod \"ingress-canary-rbkts\" (UID: \"e0a787a4-c59e-48e3-b0ac-736de6af9b84\") " pod="openshift-ingress-canary/ingress-canary-rbkts"
Jul 06 19:50:55.172502 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.172150    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rbkts"
Jul 06 19:50:55.332803 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.332762    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:50:55.332803 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.332795    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:50:55.333047 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.332798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:50:55.336969 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.336945    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 19:50:55.338345 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.338121    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 19:50:55.338345 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.338148    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2fwrm\""
Jul 06 19:50:55.338526 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.338367    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 19:50:55.338526 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.338410    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fwg6z\""
Jul 06 19:50:55.338627 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.338617    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 19:50:55.605315 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:55.605226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:55.605452 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:55.605379    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:50:55.605452 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:55.605451    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls podName:35a35874-da0b-41fe-b1d6-25bd8082e927 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:56.605433141 +0000 UTC m=+36.916888347 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls") pod "dns-default-lqm67" (UID: "35a35874-da0b-41fe-b1d6-25bd8082e927") : secret "dns-default-metrics-tls" not found
Jul 06 19:50:56.599976 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:56.599954    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rbkts"]
Jul 06 19:50:56.603744 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:50:56.603719    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a787a4_c59e_48e3_b0ac_736de6af9b84.slice/crio-8427b16c8dbfa47b72f5a6ab8773f301309b018ed0da4a3aaf89506d96111ecf WatchSource:0}: Error finding container 8427b16c8dbfa47b72f5a6ab8773f301309b018ed0da4a3aaf89506d96111ecf: Status 404 returned error can't find the container with id 8427b16c8dbfa47b72f5a6ab8773f301309b018ed0da4a3aaf89506d96111ecf
Jul 06 19:50:56.612642 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:56.612612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:56.612804 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:56.612784    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:50:56.612855 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:56.612849    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls podName:35a35874-da0b-41fe-b1d6-25bd8082e927 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:58.612829914 +0000 UTC m=+38.924285117 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls") pod "dns-default-lqm67" (UID: "35a35874-da0b-41fe-b1d6-25bd8082e927") : secret "dns-default-metrics-tls" not found
Jul 06 19:50:57.599712 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:57.599678    2577 generic.go:356] "Generic (PLEG): container finished" podID="6207540f-5268-4df4-b0fa-94cd2e4e4a44" containerID="45f6af06d000327e45d70c14d8a95ee5c42e9bdb79a4138bce7373b8af06f604" exitCode=0
Jul 06 19:50:57.599907 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:57.599765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f42z" event={"ID":"6207540f-5268-4df4-b0fa-94cd2e4e4a44","Type":"ContainerDied","Data":"45f6af06d000327e45d70c14d8a95ee5c42e9bdb79a4138bce7373b8af06f604"}
Jul 06 19:50:57.601193 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:57.600932    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rbkts" event={"ID":"e0a787a4-c59e-48e3-b0ac-736de6af9b84","Type":"ContainerStarted","Data":"8427b16c8dbfa47b72f5a6ab8773f301309b018ed0da4a3aaf89506d96111ecf"}
Jul 06 19:50:58.607778 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:58.607370    2577 generic.go:356] "Generic (PLEG): container finished" podID="6207540f-5268-4df4-b0fa-94cd2e4e4a44" containerID="346e6b30db6b526b03860c8aa0b192843c214a8601125e20abdd5c6178d595b2" exitCode=0
Jul 06 19:50:58.607778 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:58.607453    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f42z" event={"ID":"6207540f-5268-4df4-b0fa-94cd2e4e4a44","Type":"ContainerDied","Data":"346e6b30db6b526b03860c8aa0b192843c214a8601125e20abdd5c6178d595b2"}
Jul 06 19:50:58.629024 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:58.628988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:50:58.629160 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:58.629124    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:50:58.629233 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:50:58.629221    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls podName:35a35874-da0b-41fe-b1d6-25bd8082e927 nodeName:}" failed. No retries permitted until 2026-07-06 19:51:02.629200875 +0000 UTC m=+42.940656077 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls") pod "dns-default-lqm67" (UID: "35a35874-da0b-41fe-b1d6-25bd8082e927") : secret "dns-default-metrics-tls" not found
Jul 06 19:50:59.612737 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:59.612706    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f42z" event={"ID":"6207540f-5268-4df4-b0fa-94cd2e4e4a44","Type":"ContainerStarted","Data":"c953f14ba05695b61feb007e663843780e6c50767ed5d18626d9199a854c4600"}
Jul 06 19:50:59.614130 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:59.614107    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rbkts" event={"ID":"e0a787a4-c59e-48e3-b0ac-736de6af9b84","Type":"ContainerStarted","Data":"abf23b4958cd0d7923a6741f062c061bb1bfc0b6d397c5acd971ae97e09f0ab5"}
Jul 06 19:50:59.639028 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:59.638935    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8f42z" podStartSLOduration=9.713163906 podStartE2EDuration="39.638918218s" totalImagesPullingTime="29.925754312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:59.638547303 +0000 UTC m=+39.950002545" watchObservedRunningTime="2026-07-06 19:50:59.638918218 +0000 UTC m=+39.950373438"
Jul 06 19:50:59.653989 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:50:59.653951    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rbkts" podStartSLOduration=3.127928258 podStartE2EDuration="5.653938759s" totalImagesPullingTime="2.526010501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:59.653895198 +0000 UTC m=+39.965350420" watchObservedRunningTime="2026-07-06 19:50:59.653938759 +0000 UTC m=+39.965393980"
Jul 06 19:51:02.658699 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:02.658659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:51:02.659063 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:51:02.658808    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:51:02.659063 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:51:02.658875    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls podName:35a35874-da0b-41fe-b1d6-25bd8082e927 nodeName:}" failed. No retries permitted until 2026-07-06 19:51:10.658859561 +0000 UTC m=+50.970314760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls") pod "dns-default-lqm67" (UID: "35a35874-da0b-41fe-b1d6-25bd8082e927") : secret "dns-default-metrics-tls" not found
Jul 06 19:51:10.715403 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:10.715363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:51:10.715975 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:51:10.715527    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:51:10.715975 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:51:10.715620    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls podName:35a35874-da0b-41fe-b1d6-25bd8082e927 nodeName:}" failed. No retries permitted until 2026-07-06 19:51:26.715599269 +0000 UTC m=+67.027054486 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls") pod "dns-default-lqm67" (UID: "35a35874-da0b-41fe-b1d6-25bd8082e927") : secret "dns-default-metrics-tls" not found
Jul 06 19:51:21.590000 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:21.589974    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q6sq5"
Jul 06 19:51:26.024782 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.024726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:51:26.024782 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.024792    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:51:26.027527 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.027508    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 19:51:26.027527 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.027519    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 19:51:26.035874 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:51:26.035853    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 19:51:26.035961 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:51:26.035915    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs podName:17b8e220-72ef-4627-9a4e-2e57eb63aa7c nodeName:}" failed. No retries permitted until 2026-07-06 19:52:30.035898093 +0000 UTC m=+130.347353292 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs") pod "network-metrics-daemon-6dbm7" (UID: "17b8e220-72ef-4627-9a4e-2e57eb63aa7c") : secret "metrics-daemon-secret" not found
Jul 06 19:51:26.039588 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.039559    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6feda960-34dc-47d7-bd85-4cbbc95bf5ba-original-pull-secret\") pod \"global-pull-secret-syncer-8x5db\" (UID: \"6feda960-34dc-47d7-bd85-4cbbc95bf5ba\") " pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:51:26.125380 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.125331    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgnf2\" (UniqueName: \"kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2\") pod \"network-check-target-vqvgv\" (UID: \"1b321be0-8e73-4d70-91d7-e42975820cb4\") " pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:51:26.128363 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.128344    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 19:51:26.138691 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.138672    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 19:51:26.149113 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.149086    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgnf2\" (UniqueName: \"kubernetes.io/projected/1b321be0-8e73-4d70-91d7-e42975820cb4-kube-api-access-xgnf2\") pod \"network-check-target-vqvgv\" (UID: \"1b321be0-8e73-4d70-91d7-e42975820cb4\") " pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:51:26.244599 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.244552    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8x5db"
Jul 06 19:51:26.259260 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.259234    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fwg6z\""
Jul 06 19:51:26.267420 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.267401    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:51:26.377768 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.377740    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8x5db"]
Jul 06 19:51:26.381074 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:51:26.381047    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6feda960_34dc_47d7_bd85_4cbbc95bf5ba.slice/crio-a13391f0e9f163154ee5c18a411cd7fbee344d5d936a10d4ef8c33fde47b4eed WatchSource:0}: Error finding container a13391f0e9f163154ee5c18a411cd7fbee344d5d936a10d4ef8c33fde47b4eed: Status 404 returned error can't find the container with id a13391f0e9f163154ee5c18a411cd7fbee344d5d936a10d4ef8c33fde47b4eed
Jul 06 19:51:26.393335 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.393315    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vqvgv"]
Jul 06 19:51:26.396074 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:51:26.396047    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b321be0_8e73_4d70_91d7_e42975820cb4.slice/crio-8a433512bbed80e02eb576136579dff90bd3b3cf918a049a62138afdbd6a7e9a WatchSource:0}: Error finding container 8a433512bbed80e02eb576136579dff90bd3b3cf918a049a62138afdbd6a7e9a: Status 404 returned error can't find the container with id 8a433512bbed80e02eb576136579dff90bd3b3cf918a049a62138afdbd6a7e9a
Jul 06 19:51:26.677427 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.677395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vqvgv" event={"ID":"1b321be0-8e73-4d70-91d7-e42975820cb4","Type":"ContainerStarted","Data":"8a433512bbed80e02eb576136579dff90bd3b3cf918a049a62138afdbd6a7e9a"}
Jul 06 19:51:26.678388 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.678365    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8x5db" event={"ID":"6feda960-34dc-47d7-bd85-4cbbc95bf5ba","Type":"ContainerStarted","Data":"a13391f0e9f163154ee5c18a411cd7fbee344d5d936a10d4ef8c33fde47b4eed"}
Jul 06 19:51:26.730787 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:26.730757    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:51:26.730920 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:51:26.730876    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:51:26.730958 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:51:26.730929    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls podName:35a35874-da0b-41fe-b1d6-25bd8082e927 nodeName:}" failed. No retries permitted until 2026-07-06 19:51:58.730913398 +0000 UTC m=+99.042368597 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls") pod "dns-default-lqm67" (UID: "35a35874-da0b-41fe-b1d6-25bd8082e927") : secret "dns-default-metrics-tls" not found
Jul 06 19:51:29.317187 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.317116    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"]
Jul 06 19:51:29.337796 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.331485    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"]
Jul 06 19:51:29.337796 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.332656    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:29.337796 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.337072    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 19:51:29.343994 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.343437    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 19:51:29.343994 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.343591    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 06 19:51:29.343994 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.343714    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 19:51:29.350166 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.350141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6e5afbc-3972-42da-90b9-2aed6c8ac698-tmp\") pod \"klusterlet-addon-workmgr-7947dcc55-8sgrv\" (UID: \"d6e5afbc-3972-42da-90b9-2aed6c8ac698\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:29.350322 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.350192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d6e5afbc-3972-42da-90b9-2aed6c8ac698-klusterlet-config\") pod \"klusterlet-addon-workmgr-7947dcc55-8sgrv\" (UID: \"d6e5afbc-3972-42da-90b9-2aed6c8ac698\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:29.350322 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.350222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjg6c\" (UniqueName: \"kubernetes.io/projected/d6e5afbc-3972-42da-90b9-2aed6c8ac698-kube-api-access-hjg6c\") pod \"klusterlet-addon-workmgr-7947dcc55-8sgrv\" (UID: \"d6e5afbc-3972-42da-90b9-2aed6c8ac698\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:29.351427 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.351404    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"]
Jul 06 19:51:29.351517 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.351448    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"]
Jul 06 19:51:29.351568 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.351545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.354213 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.354194    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 06 19:51:29.354542 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.354526    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 06 19:51:29.354639 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.354597    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 06 19:51:29.355012 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.354995    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 06 19:51:29.450892 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.450855    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-hub\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.450892 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.450905    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.451153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.450938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/83704fd5-cfad-471d-a5af-3211a50e47e6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.451153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.451016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.451153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.451082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkq4f\" (UniqueName: \"kubernetes.io/projected/83704fd5-cfad-471d-a5af-3211a50e47e6-kube-api-access-kkq4f\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.451153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.451121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6e5afbc-3972-42da-90b9-2aed6c8ac698-tmp\") pod \"klusterlet-addon-workmgr-7947dcc55-8sgrv\" (UID: \"d6e5afbc-3972-42da-90b9-2aed6c8ac698\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:29.451153 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.451149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d6e5afbc-3972-42da-90b9-2aed6c8ac698-klusterlet-config\") pod \"klusterlet-addon-workmgr-7947dcc55-8sgrv\" (UID: \"d6e5afbc-3972-42da-90b9-2aed6c8ac698\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:29.451389 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.451188    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-ca\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.451389 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.451220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjg6c\" (UniqueName: \"kubernetes.io/projected/d6e5afbc-3972-42da-90b9-2aed6c8ac698-kube-api-access-hjg6c\") pod \"klusterlet-addon-workmgr-7947dcc55-8sgrv\" (UID: \"d6e5afbc-3972-42da-90b9-2aed6c8ac698\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:29.451583 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.451560    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6e5afbc-3972-42da-90b9-2aed6c8ac698-tmp\") pod \"klusterlet-addon-workmgr-7947dcc55-8sgrv\" (UID: \"d6e5afbc-3972-42da-90b9-2aed6c8ac698\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:29.454242 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.454223    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d6e5afbc-3972-42da-90b9-2aed6c8ac698-klusterlet-config\") pod \"klusterlet-addon-workmgr-7947dcc55-8sgrv\" (UID: \"d6e5afbc-3972-42da-90b9-2aed6c8ac698\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:29.460049 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.460020    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjg6c\" (UniqueName: \"kubernetes.io/projected/d6e5afbc-3972-42da-90b9-2aed6c8ac698-kube-api-access-hjg6c\") pod \"klusterlet-addon-workmgr-7947dcc55-8sgrv\" (UID: \"d6e5afbc-3972-42da-90b9-2aed6c8ac698\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:29.551543 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.551508    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-hub\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.551543 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.551547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.551759 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.551571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/83704fd5-cfad-471d-a5af-3211a50e47e6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.551759 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.551613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.551839 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.551784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkq4f\" (UniqueName: \"kubernetes.io/projected/83704fd5-cfad-471d-a5af-3211a50e47e6-kube-api-access-kkq4f\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.551890 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.551849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-ca\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.552383 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.552356    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/83704fd5-cfad-471d-a5af-3211a50e47e6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.554290 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.554239    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.554420 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.554357    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.554691 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.554671    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-ca\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.554771 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.554750    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/83704fd5-cfad-471d-a5af-3211a50e47e6-hub\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.560370 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.560349    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkq4f\" (UniqueName: \"kubernetes.io/projected/83704fd5-cfad-471d-a5af-3211a50e47e6-kube-api-access-kkq4f\") pod \"cluster-proxy-proxy-agent-7cc8cfdd95-2xj87\" (UID: \"83704fd5-cfad-471d-a5af-3211a50e47e6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:29.654484 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.654447    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:29.674910 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:29.674887    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:51:31.095506 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:31.095404    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"]
Jul 06 19:51:31.099143 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:51:31.099113    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83704fd5_cfad_471d_a5af_3211a50e47e6.slice/crio-5f09bcf8cbe08502a613eaef6409f4a8f3772fb2cf38a6ccd74cce4ef153c359 WatchSource:0}: Error finding container 5f09bcf8cbe08502a613eaef6409f4a8f3772fb2cf38a6ccd74cce4ef153c359: Status 404 returned error can't find the container with id 5f09bcf8cbe08502a613eaef6409f4a8f3772fb2cf38a6ccd74cce4ef153c359
Jul 06 19:51:31.110225 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:31.110204    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"]
Jul 06 19:51:31.113684 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:51:31.113656    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e5afbc_3972_42da_90b9_2aed6c8ac698.slice/crio-ba68988ee2be2a03a73ba3d013f43210ef1401297ef81a5992a8d1187e615e0f WatchSource:0}: Error finding container ba68988ee2be2a03a73ba3d013f43210ef1401297ef81a5992a8d1187e615e0f: Status 404 returned error can't find the container with id ba68988ee2be2a03a73ba3d013f43210ef1401297ef81a5992a8d1187e615e0f
Jul 06 19:51:31.693698 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:31.693659    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8x5db" event={"ID":"6feda960-34dc-47d7-bd85-4cbbc95bf5ba","Type":"ContainerStarted","Data":"c11edd06903125d3a072f1fc50fe8b8674b8e8188313bece8ce4b3003e61472c"}
Jul 06 19:51:31.694978 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:31.694950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv" event={"ID":"d6e5afbc-3972-42da-90b9-2aed6c8ac698","Type":"ContainerStarted","Data":"ba68988ee2be2a03a73ba3d013f43210ef1401297ef81a5992a8d1187e615e0f"}
Jul 06 19:51:31.696167 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:31.696131    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" event={"ID":"83704fd5-cfad-471d-a5af-3211a50e47e6","Type":"ContainerStarted","Data":"5f09bcf8cbe08502a613eaef6409f4a8f3772fb2cf38a6ccd74cce4ef153c359"}
Jul 06 19:51:31.697706 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:31.697688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vqvgv" event={"ID":"1b321be0-8e73-4d70-91d7-e42975820cb4","Type":"ContainerStarted","Data":"888be58a1bb63e09efc0390502e5d59f16db7d3ee12bcfefd4d141e692b7fe19"}
Jul 06 19:51:31.697836 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:31.697823    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:51:31.724038 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:31.723986    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8x5db" podStartSLOduration=67.136997765 podStartE2EDuration="1m11.723967772s" totalImagesPullingTime="4.586970007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:51:31.722998593 +0000 UTC m=+72.034453815" watchObservedRunningTime="2026-07-06 19:51:31.723967772 +0000 UTC m=+72.035422995"
Jul 06 19:51:31.739307 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:31.739258    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vqvgv" podStartSLOduration=67.178157155 podStartE2EDuration="1m11.739243594s" totalImagesPullingTime="4.561086439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:51:31.738313858 +0000 UTC m=+72.049769078" watchObservedRunningTime="2026-07-06 19:51:31.739243594 +0000 UTC m=+72.050698806"
Jul 06 19:51:37.719850 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:37.719809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv" event={"ID":"d6e5afbc-3972-42da-90b9-2aed6c8ac698","Type":"ContainerStarted","Data":"4206101f6d65fce85908082892db16a5bbc2199b0c4b5b85840a80c751b84302"}
Jul 06 19:51:37.720335 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:37.720027    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:37.721851 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:37.721829    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:51:37.722094 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:37.722073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" event={"ID":"83704fd5-cfad-471d-a5af-3211a50e47e6","Type":"ContainerStarted","Data":"84038b1b4f2c92055829f89eb0788807d66b9c0823c9a9ed6dd00679923138e5"}
Jul 06 19:51:37.722141 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:37.722104    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" event={"ID":"83704fd5-cfad-471d-a5af-3211a50e47e6","Type":"ContainerStarted","Data":"df558f850baf794b1461fbce28b5b93bb44601675e4b68dc3c229e01fa74a995"}
Jul 06 19:51:37.722141 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:37.722116    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" event={"ID":"83704fd5-cfad-471d-a5af-3211a50e47e6","Type":"ContainerStarted","Data":"141f33f027a3fabbc0a48ccb4284d5925b5ea38f3dd774bf1d2e53dfc15ba4f8"}
Jul 06 19:51:37.737377 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:37.737331    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv" podStartSLOduration=2.58696203 podStartE2EDuration="8.737315914s" totalImagesPullingTime="6.150353884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:51:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:51:37.73651246 +0000 UTC m=+78.047967681" watchObservedRunningTime="2026-07-06 19:51:37.737315914 +0000 UTC m=+78.048771134"
Jul 06 19:51:37.754344 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:37.754304    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" podStartSLOduration=2.591328002 podStartE2EDuration="8.75429227s" totalImagesPullingTime="6.162964268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:51:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:51:37.753744242 +0000 UTC m=+78.065199464" watchObservedRunningTime="2026-07-06 19:51:37.75429227 +0000 UTC m=+78.065747491"
Jul 06 19:51:58.766019 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:51:58.765901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:51:58.766416 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:51:58.766046    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:51:58.766416 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:51:58.766111    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls podName:35a35874-da0b-41fe-b1d6-25bd8082e927 nodeName:}" failed. No retries permitted until 2026-07-06 19:53:02.766092576 +0000 UTC m=+163.077547781 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls") pod "dns-default-lqm67" (UID: "35a35874-da0b-41fe-b1d6-25bd8082e927") : secret "dns-default-metrics-tls" not found
Jul 06 19:52:02.703624 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:52:02.703587    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vqvgv"
Jul 06 19:52:30.090098 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:52:30.090043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:52:30.090635 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:52:30.090206    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 19:52:30.090635 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:52:30.090273    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs podName:17b8e220-72ef-4627-9a4e-2e57eb63aa7c nodeName:}" failed. No retries permitted until 2026-07-06 19:54:32.090257218 +0000 UTC m=+252.401712416 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs") pod "network-metrics-daemon-6dbm7" (UID: "17b8e220-72ef-4627-9a4e-2e57eb63aa7c") : secret "metrics-daemon-secret" not found
Jul 06 19:52:57.878816 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:52:57.878762    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lqm67" podUID="35a35874-da0b-41fe-b1d6-25bd8082e927"
Jul 06 19:52:57.949234 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:52:57.949203    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lqm67"
Jul 06 19:52:58.350965 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:52:58.350878    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:53:02.816360 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:53:02.816324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:53:02.816844 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:53:02.816477    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 19:53:02.816844 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:53:02.816555    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls podName:35a35874-da0b-41fe-b1d6-25bd8082e927 nodeName:}" failed. No retries permitted until 2026-07-06 19:55:04.81654023 +0000 UTC m=+285.127995429 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls") pod "dns-default-lqm67" (UID: "35a35874-da0b-41fe-b1d6-25bd8082e927") : secret "dns-default-metrics-tls" not found
Jul 06 19:53:12.332310 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:53:12.332221    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:53:37.721116 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:53:37.721064    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv" podUID="d6e5afbc-3972-42da-90b9-2aed6c8ac698" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused"
Jul 06 19:53:38.058346 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:53:38.058265    2577 generic.go:356] "Generic (PLEG): container finished" podID="d6e5afbc-3972-42da-90b9-2aed6c8ac698" containerID="4206101f6d65fce85908082892db16a5bbc2199b0c4b5b85840a80c751b84302" exitCode=1
Jul 06 19:53:38.058346 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:53:38.058324    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv" event={"ID":"d6e5afbc-3972-42da-90b9-2aed6c8ac698","Type":"ContainerDied","Data":"4206101f6d65fce85908082892db16a5bbc2199b0c4b5b85840a80c751b84302"}
Jul 06 19:53:38.058620 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:53:38.058606    2577 scope.go:117] "RemoveContainer" containerID="4206101f6d65fce85908082892db16a5bbc2199b0c4b5b85840a80c751b84302"
Jul 06 19:53:39.062542 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:53:39.062510    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv" event={"ID":"d6e5afbc-3972-42da-90b9-2aed6c8ac698","Type":"ContainerStarted","Data":"0bf2ce92d86574b7d9d7fd41f2dac320913dc6e1d52f84e1689ac53fa25110eb"}
Jul 06 19:53:39.062991 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:53:39.062789    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:53:39.063434 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:53:39.063414    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7947dcc55-8sgrv"
Jul 06 19:54:32.134501 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:32.134467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:54:32.134986 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:32.134618    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 19:54:32.134986 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:32.134675    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs podName:17b8e220-72ef-4627-9a4e-2e57eb63aa7c nodeName:}" failed. No retries permitted until 2026-07-06 19:56:34.134659752 +0000 UTC m=+374.446114954 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs") pod "network-metrics-daemon-6dbm7" (UID: "17b8e220-72ef-4627-9a4e-2e57eb63aa7c") : secret "metrics-daemon-secret" not found
Jul 06 19:54:50.767588 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.767507    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-79s77"]
Jul 06 19:54:50.769773 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.769753    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.774929 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.774908    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 19:54:50.776103 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.776079    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lpwlh\""
Jul 06 19:54:50.776230 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.776128    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 19:54:50.776230 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.776136    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 19:54:50.776230 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.776137    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 19:54:50.782449 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.782427    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-79s77"]
Jul 06 19:54:50.863971 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.863935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b122086a-cb4b-4021-80ab-391c8097b4a1-crio-socket\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.863971 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.863971    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b122086a-cb4b-4021-80ab-391c8097b4a1-data-volume\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.864227 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.864002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8bhh\" (UniqueName: \"kubernetes.io/projected/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-api-access-s8bhh\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.864227 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.864043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.864227 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.864084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.964871 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.964838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.964988 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.964893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.964988 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.964923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b122086a-cb4b-4021-80ab-391c8097b4a1-crio-socket\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.964988 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.964939    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b122086a-cb4b-4021-80ab-391c8097b4a1-data-volume\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.964988 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:50.964961    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 19:54:50.964988 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.964968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8bhh\" (UniqueName: \"kubernetes.io/projected/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-api-access-s8bhh\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.965170 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:50.965035    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm podName:b122086a-cb4b-4021-80ab-391c8097b4a1 nodeName:}" failed. No retries permitted until 2026-07-06 19:54:51.465013238 +0000 UTC m=+271.776468457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-79s77" (UID: "b122086a-cb4b-4021-80ab-391c8097b4a1") : configmap "kube-rbac-proxy" not found
Jul 06 19:54:50.965170 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.965047    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b122086a-cb4b-4021-80ab-391c8097b4a1-crio-socket\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.965170 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:50.965113    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:54:50.965170 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:50.965163    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls podName:b122086a-cb4b-4021-80ab-391c8097b4a1 nodeName:}" failed. No retries permitted until 2026-07-06 19:54:51.465148132 +0000 UTC m=+271.776603348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-79s77" (UID: "b122086a-cb4b-4021-80ab-391c8097b4a1") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:54:50.965352 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.965331    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b122086a-cb4b-4021-80ab-391c8097b4a1-data-volume\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:50.973627 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:50.973598    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8bhh\" (UniqueName: \"kubernetes.io/projected/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-api-access-s8bhh\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:51.468570 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:51.468531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:51.468742 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:51.468601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:51.468742 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:51.468665    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 19:54:51.468742 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:51.468687    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:54:51.468742 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:51.468743    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls podName:b122086a-cb4b-4021-80ab-391c8097b4a1 nodeName:}" failed. No retries permitted until 2026-07-06 19:54:52.468721712 +0000 UTC m=+272.780176914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-79s77" (UID: "b122086a-cb4b-4021-80ab-391c8097b4a1") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:54:51.468881 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:51.468760    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm podName:b122086a-cb4b-4021-80ab-391c8097b4a1 nodeName:}" failed. No retries permitted until 2026-07-06 19:54:52.468751335 +0000 UTC m=+272.780206541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-79s77" (UID: "b122086a-cb4b-4021-80ab-391c8097b4a1") : configmap "kube-rbac-proxy" not found
Jul 06 19:54:52.476687 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:52.476648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:52.477077 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:52.476704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:52.477077 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:52.476778    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 19:54:52.477077 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:52.476795    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:54:52.477077 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:52.476845    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls podName:b122086a-cb4b-4021-80ab-391c8097b4a1 nodeName:}" failed. No retries permitted until 2026-07-06 19:54:54.476832161 +0000 UTC m=+274.788287361 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-79s77" (UID: "b122086a-cb4b-4021-80ab-391c8097b4a1") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:54:52.477077 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:52.476857    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm podName:b122086a-cb4b-4021-80ab-391c8097b4a1 nodeName:}" failed. No retries permitted until 2026-07-06 19:54:54.47685179 +0000 UTC m=+274.788306988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-79s77" (UID: "b122086a-cb4b-4021-80ab-391c8097b4a1") : configmap "kube-rbac-proxy" not found
Jul 06 19:54:52.734980 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:52.734910    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qzw6h_f2473042-4162-458d-8969-ae7e8668b0b3/dns-node-resolver/0.log"
Jul 06 19:54:53.535697 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:53.535671    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-29qdv_509ae1fb-ff23-4a78-acb5-5f55e5900408/node-ca/0.log"
Jul 06 19:54:54.491208 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:54.491155    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:54.491402 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:54.491255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:54.491402 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:54.491303    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:54:54.491402 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:54.491332    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 19:54:54.491402 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:54.491368    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls podName:b122086a-cb4b-4021-80ab-391c8097b4a1 nodeName:}" failed. No retries permitted until 2026-07-06 19:54:58.49135261 +0000 UTC m=+278.802807811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-79s77" (UID: "b122086a-cb4b-4021-80ab-391c8097b4a1") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:54:54.491402 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:54.491384    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm podName:b122086a-cb4b-4021-80ab-391c8097b4a1 nodeName:}" failed. No retries permitted until 2026-07-06 19:54:58.491377586 +0000 UTC m=+278.802832784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-79s77" (UID: "b122086a-cb4b-4021-80ab-391c8097b4a1") : configmap "kube-rbac-proxy" not found
Jul 06 19:54:54.536001 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:54.535979    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rbkts_e0a787a4-c59e-48e3-b0ac-736de6af9b84/serve-healthcheck-canary/0.log"
Jul 06 19:54:58.518766 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:58.518714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:58.519166 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:54:58.518813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:54:58.519166 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:58.518875    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:54:58.519166 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:58.518895    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 19:54:58.519166 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:58.518941    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls podName:b122086a-cb4b-4021-80ab-391c8097b4a1 nodeName:}" failed. No retries permitted until 2026-07-06 19:55:06.518926227 +0000 UTC m=+286.830381435 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-79s77" (UID: "b122086a-cb4b-4021-80ab-391c8097b4a1") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:54:58.519166 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:54:58.518956    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm podName:b122086a-cb4b-4021-80ab-391c8097b4a1 nodeName:}" failed. No retries permitted until 2026-07-06 19:55:06.518950711 +0000 UTC m=+286.830405915 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-79s77" (UID: "b122086a-cb4b-4021-80ab-391c8097b4a1") : configmap "kube-rbac-proxy" not found
Jul 06 19:55:00.949629 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:55:00.949587    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lqm67" podUID="35a35874-da0b-41fe-b1d6-25bd8082e927"
Jul 06 19:55:01.295953 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:01.295877    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lqm67"
Jul 06 19:55:04.868741 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:04.868700    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:55:04.871066 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:04.871043    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35a35874-da0b-41fe-b1d6-25bd8082e927-metrics-tls\") pod \"dns-default-lqm67\" (UID: \"35a35874-da0b-41fe-b1d6-25bd8082e927\") " pod="openshift-dns/dns-default-lqm67"
Jul 06 19:55:04.899630 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:04.899603    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-45vdg\""
Jul 06 19:55:04.907728 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:04.907707    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lqm67"
Jul 06 19:55:05.024853 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:05.024820    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lqm67"]
Jul 06 19:55:05.027976 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:55:05.027951    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35a35874_da0b_41fe_b1d6_25bd8082e927.slice/crio-28c999ce178249cf7acf7b1c2683bcfcd8ed46990669febf55ecfcedad4a2e7d WatchSource:0}: Error finding container 28c999ce178249cf7acf7b1c2683bcfcd8ed46990669febf55ecfcedad4a2e7d: Status 404 returned error can't find the container with id 28c999ce178249cf7acf7b1c2683bcfcd8ed46990669febf55ecfcedad4a2e7d
Jul 06 19:55:05.307399 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:05.307364    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lqm67" event={"ID":"35a35874-da0b-41fe-b1d6-25bd8082e927","Type":"ContainerStarted","Data":"28c999ce178249cf7acf7b1c2683bcfcd8ed46990669febf55ecfcedad4a2e7d"}
Jul 06 19:55:06.581623 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:06.581590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:55:06.581936 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:06.581639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:55:06.581936 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:55:06.581735    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 19:55:06.581936 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:55:06.581791    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls podName:b122086a-cb4b-4021-80ab-391c8097b4a1 nodeName:}" failed. No retries permitted until 2026-07-06 19:55:22.58177512 +0000 UTC m=+302.893230336 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-79s77" (UID: "b122086a-cb4b-4021-80ab-391c8097b4a1") : secret "insights-runtime-extractor-tls" not found
Jul 06 19:55:06.582066 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:06.582045    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b122086a-cb4b-4021-80ab-391c8097b4a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:55:07.315609 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:07.315578    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lqm67" event={"ID":"35a35874-da0b-41fe-b1d6-25bd8082e927","Type":"ContainerStarted","Data":"22936043b7dcfcad7c800084efa6441918aadc43a5ebde66c3ac6f9d38207323"}
Jul 06 19:55:07.315797 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:07.315625    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lqm67" event={"ID":"35a35874-da0b-41fe-b1d6-25bd8082e927","Type":"ContainerStarted","Data":"c9b0bdaab41426f411244d45f83a9c447f75c27765c54580d6d95bc55456e7b2"}
Jul 06 19:55:07.315797 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:07.315721    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lqm67"
Jul 06 19:55:07.341513 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:07.341470    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lqm67" podStartSLOduration=251.95139456 podStartE2EDuration="4m13.341453494s" totalImagesPullingTime="1.390058934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:07.340335107 +0000 UTC m=+287.651790319" watchObservedRunningTime="2026-07-06 19:55:07.341453494 +0000 UTC m=+287.652908718"
Jul 06 19:55:09.676664 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:09.676628    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" podUID="83704fd5-cfad-471d-a5af-3211a50e47e6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 19:55:12.998574 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:12.998541    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7887966bdc-54bwk"]
Jul 06 19:55:13.001290 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.001271    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.004037 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.004007    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 19:55:13.004131 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.004009    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 19:55:13.004377 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.004360    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 19:55:13.005270 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.005256    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s7fls\""
Jul 06 19:55:13.011066 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.011049    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 19:55:13.017853 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.017833    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7887966bdc-54bwk"]
Jul 06 19:55:13.128764 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.128735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6418cafc-b74f-49ae-9c50-4537ac11d2ea-ca-trust-extracted\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.128957 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.128775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v2kx\" (UniqueName: \"kubernetes.io/projected/6418cafc-b74f-49ae-9c50-4537ac11d2ea-kube-api-access-4v2kx\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.128957 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.128838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6418cafc-b74f-49ae-9c50-4537ac11d2ea-installation-pull-secrets\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.128957 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.128885    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6418cafc-b74f-49ae-9c50-4537ac11d2ea-bound-sa-token\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.128957 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.128929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6418cafc-b74f-49ae-9c50-4537ac11d2ea-image-registry-private-configuration\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.128957 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.128953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6418cafc-b74f-49ae-9c50-4537ac11d2ea-trusted-ca\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.129151 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.128975    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6418cafc-b74f-49ae-9c50-4537ac11d2ea-registry-tls\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.129151 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.128992    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6418cafc-b74f-49ae-9c50-4537ac11d2ea-registry-certificates\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.229337 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.229296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6418cafc-b74f-49ae-9c50-4537ac11d2ea-trusted-ca\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.229525 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.229345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6418cafc-b74f-49ae-9c50-4537ac11d2ea-registry-tls\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.229525 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.229451    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6418cafc-b74f-49ae-9c50-4537ac11d2ea-registry-certificates\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.229525 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.229487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6418cafc-b74f-49ae-9c50-4537ac11d2ea-ca-trust-extracted\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.229525 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.229524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v2kx\" (UniqueName: \"kubernetes.io/projected/6418cafc-b74f-49ae-9c50-4537ac11d2ea-kube-api-access-4v2kx\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.229738 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.229570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6418cafc-b74f-49ae-9c50-4537ac11d2ea-installation-pull-secrets\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.229738 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.229609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6418cafc-b74f-49ae-9c50-4537ac11d2ea-bound-sa-token\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.229738 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.229660    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6418cafc-b74f-49ae-9c50-4537ac11d2ea-image-registry-private-configuration\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.230011 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.229984    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6418cafc-b74f-49ae-9c50-4537ac11d2ea-ca-trust-extracted\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.230399 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.230375    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6418cafc-b74f-49ae-9c50-4537ac11d2ea-registry-certificates\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.230727 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.230703    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6418cafc-b74f-49ae-9c50-4537ac11d2ea-trusted-ca\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.231847 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.231825    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6418cafc-b74f-49ae-9c50-4537ac11d2ea-registry-tls\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.232123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.232107    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6418cafc-b74f-49ae-9c50-4537ac11d2ea-image-registry-private-configuration\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.232123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.232113    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6418cafc-b74f-49ae-9c50-4537ac11d2ea-installation-pull-secrets\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.239645 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.239619    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v2kx\" (UniqueName: \"kubernetes.io/projected/6418cafc-b74f-49ae-9c50-4537ac11d2ea-kube-api-access-4v2kx\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.239720 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.239711    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6418cafc-b74f-49ae-9c50-4537ac11d2ea-bound-sa-token\") pod \"image-registry-7887966bdc-54bwk\" (UID: \"6418cafc-b74f-49ae-9c50-4537ac11d2ea\") " pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.309685 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.309604    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:13.433101 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:13.433071    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7887966bdc-54bwk"]
Jul 06 19:55:13.436765 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:55:13.436739    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6418cafc_b74f_49ae_9c50_4537ac11d2ea.slice/crio-eeb4864ea42e4ac59ff0e959200ae0492b6f3afba2851f7254cdd4e0c975137c WatchSource:0}: Error finding container eeb4864ea42e4ac59ff0e959200ae0492b6f3afba2851f7254cdd4e0c975137c: Status 404 returned error can't find the container with id eeb4864ea42e4ac59ff0e959200ae0492b6f3afba2851f7254cdd4e0c975137c
Jul 06 19:55:14.338467 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:14.338424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7887966bdc-54bwk" event={"ID":"6418cafc-b74f-49ae-9c50-4537ac11d2ea","Type":"ContainerStarted","Data":"2b859d51cb24eeeb48ea5ece714cf535bf3977a9a23d6cb3f223ad7adb588b77"}
Jul 06 19:55:14.338946 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:14.338514    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7887966bdc-54bwk" event={"ID":"6418cafc-b74f-49ae-9c50-4537ac11d2ea","Type":"ContainerStarted","Data":"eeb4864ea42e4ac59ff0e959200ae0492b6f3afba2851f7254cdd4e0c975137c"}
Jul 06 19:55:14.339136 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:14.339113    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:14.363266 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:14.363219    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7887966bdc-54bwk" podStartSLOduration=2.363203502 podStartE2EDuration="2.363203502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:14.362795039 +0000 UTC m=+294.674250260" watchObservedRunningTime="2026-07-06 19:55:14.363203502 +0000 UTC m=+294.674658722"
Jul 06 19:55:15.332687 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:55:15.332633    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6dbm7" podUID="17b8e220-72ef-4627-9a4e-2e57eb63aa7c"
Jul 06 19:55:17.321614 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:17.321587    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lqm67"
Jul 06 19:55:19.675869 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:19.675829    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" podUID="83704fd5-cfad-471d-a5af-3211a50e47e6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 19:55:20.138347 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:20.138323    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 19:55:22.608331 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:22.608282    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:55:22.610671 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:22.610633    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b122086a-cb4b-4021-80ab-391c8097b4a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79s77\" (UID: \"b122086a-cb4b-4021-80ab-391c8097b4a1\") " pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:55:22.881337 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:22.881313    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lpwlh\""
Jul 06 19:55:22.889351 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:22.889328    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-79s77"
Jul 06 19:55:23.013085 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:23.013010    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-79s77"]
Jul 06 19:55:23.016662 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:55:23.016628    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb122086a_cb4b_4021_80ab_391c8097b4a1.slice/crio-87e32d15b7969a553977aca6fa494146ddb99f42322c76255cc7d85773e24eb9 WatchSource:0}: Error finding container 87e32d15b7969a553977aca6fa494146ddb99f42322c76255cc7d85773e24eb9: Status 404 returned error can't find the container with id 87e32d15b7969a553977aca6fa494146ddb99f42322c76255cc7d85773e24eb9
Jul 06 19:55:23.018070 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:23.018050    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 19:55:23.363723 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:23.363638    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-79s77" event={"ID":"b122086a-cb4b-4021-80ab-391c8097b4a1","Type":"ContainerStarted","Data":"87e32d15b7969a553977aca6fa494146ddb99f42322c76255cc7d85773e24eb9"}
Jul 06 19:55:27.331873 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:27.331776    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:55:27.377544 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:27.377514    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-79s77" event={"ID":"b122086a-cb4b-4021-80ab-391c8097b4a1","Type":"ContainerStarted","Data":"8a92fb7e65e65555db5fbe6062436e0e9ffc0d0128df4cd8b488f812ccfea042"}
Jul 06 19:55:28.382866 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:28.382825    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-79s77" event={"ID":"b122086a-cb4b-4021-80ab-391c8097b4a1","Type":"ContainerStarted","Data":"d54aae365fc4ad3fb47c4d9c92d9acb529f6e5fadae4001e895fbac1cbde227c"}
Jul 06 19:55:29.676259 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:29.676219    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" podUID="83704fd5-cfad-471d-a5af-3211a50e47e6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 19:55:29.676678 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:29.676293    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87"
Jul 06 19:55:29.676851 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:29.676819    2577 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"84038b1b4f2c92055829f89eb0788807d66b9c0823c9a9ed6dd00679923138e5"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 06 19:55:29.676897 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:29.676873    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" podUID="83704fd5-cfad-471d-a5af-3211a50e47e6" containerName="service-proxy" containerID="cri-o://84038b1b4f2c92055829f89eb0788807d66b9c0823c9a9ed6dd00679923138e5" gracePeriod=30
Jul 06 19:55:30.335763 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.335740    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h7nr5"]
Jul 06 19:55:30.338773 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.338748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.341212 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.341186    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 19:55:30.341734 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.341713    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 19:55:30.341879 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.341768    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 19:55:30.341879 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.341793    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 19:55:30.342574 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.342323    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-td258\""
Jul 06 19:55:30.342574 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.342388    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 19:55:30.342574 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.342511    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 19:55:30.373915 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.373885    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-textfile\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.374057 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.373935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.374057 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.373974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2407df6e-673c-4112-a05d-2d677966703b-metrics-client-ca\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.374057 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.373998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2407df6e-673c-4112-a05d-2d677966703b-sys\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.374057 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.374017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-wtmp\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.374207 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.374076    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8slj\" (UniqueName: \"kubernetes.io/projected/2407df6e-673c-4112-a05d-2d677966703b-kube-api-access-v8slj\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.374207 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.374143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2407df6e-673c-4112-a05d-2d677966703b-root\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.374207 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.374190    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-accelerators-collector-config\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.374303 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.374232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-tls\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.391096 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.391068    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-79s77" event={"ID":"b122086a-cb4b-4021-80ab-391c8097b4a1","Type":"ContainerStarted","Data":"eca41a482824a354bccc1f9394b35ca5276b6576eb27a4506abcd01749d56638"}
Jul 06 19:55:30.393281 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.393252    2577 generic.go:356] "Generic (PLEG): container finished" podID="83704fd5-cfad-471d-a5af-3211a50e47e6" containerID="84038b1b4f2c92055829f89eb0788807d66b9c0823c9a9ed6dd00679923138e5" exitCode=2
Jul 06 19:55:30.393384 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.393291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" event={"ID":"83704fd5-cfad-471d-a5af-3211a50e47e6","Type":"ContainerDied","Data":"84038b1b4f2c92055829f89eb0788807d66b9c0823c9a9ed6dd00679923138e5"}
Jul 06 19:55:30.393384 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.393319    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cc8cfdd95-2xj87" event={"ID":"83704fd5-cfad-471d-a5af-3211a50e47e6","Type":"ContainerStarted","Data":"eeb93caadb62293fca391dd918ae589ba18ffce213ba1c691314ebbc30f3d8c7"}
Jul 06 19:55:30.412099 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.412058    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-79s77" podStartSLOduration=33.246512536 podStartE2EDuration="40.412044852s" totalImagesPullingTime="7.165532316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:54:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:30.411732681 +0000 UTC m=+310.723187903" watchObservedRunningTime="2026-07-06 19:55:30.412044852 +0000 UTC m=+310.723500073"
Jul 06 19:55:30.474991 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.474958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-tls\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475151 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-textfile\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475151 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:55:30.475116    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 19:55:30.475299 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475196    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475299 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:55:30.475208    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-tls podName:2407df6e-673c-4112-a05d-2d677966703b nodeName:}" failed. No retries permitted until 2026-07-06 19:55:30.975163947 +0000 UTC m=+311.286619152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-tls") pod "node-exporter-h7nr5" (UID: "2407df6e-673c-4112-a05d-2d677966703b") : secret "node-exporter-tls" not found
Jul 06 19:55:30.475411 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2407df6e-673c-4112-a05d-2d677966703b-metrics-client-ca\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475411 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475334    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-textfile\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475514 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2407df6e-673c-4112-a05d-2d677966703b-sys\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475514 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475505    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2407df6e-673c-4112-a05d-2d677966703b-sys\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475614 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-wtmp\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475715 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8slj\" (UniqueName: \"kubernetes.io/projected/2407df6e-673c-4112-a05d-2d677966703b-kube-api-access-v8slj\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475803 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475692    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-wtmp\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475803 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2407df6e-673c-4112-a05d-2d677966703b-root\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475803 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-accelerators-collector-config\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.475911 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.475799    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2407df6e-673c-4112-a05d-2d677966703b-root\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.476076 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.476060    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2407df6e-673c-4112-a05d-2d677966703b-metrics-client-ca\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.476146 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.476130    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-accelerators-collector-config\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.477619 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.477601    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.484369 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.484351    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8slj\" (UniqueName: \"kubernetes.io/projected/2407df6e-673c-4112-a05d-2d677966703b-kube-api-access-v8slj\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.978795 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.978748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-tls\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:30.980958 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:30.980935    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2407df6e-673c-4112-a05d-2d677966703b-node-exporter-tls\") pod \"node-exporter-h7nr5\" (UID: \"2407df6e-673c-4112-a05d-2d677966703b\") " pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:31.248946 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:31.248867    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h7nr5"
Jul 06 19:55:31.259901 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:55:31.259872    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2407df6e_673c_4112_a05d_2d677966703b.slice/crio-9428d6fa4986d818927c93f0a8d70f169b9a52ecfdd8bc9bfccd17d22879dc34 WatchSource:0}: Error finding container 9428d6fa4986d818927c93f0a8d70f169b9a52ecfdd8bc9bfccd17d22879dc34: Status 404 returned error can't find the container with id 9428d6fa4986d818927c93f0a8d70f169b9a52ecfdd8bc9bfccd17d22879dc34
Jul 06 19:55:31.397140 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:31.397106    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h7nr5" event={"ID":"2407df6e-673c-4112-a05d-2d677966703b","Type":"ContainerStarted","Data":"9428d6fa4986d818927c93f0a8d70f169b9a52ecfdd8bc9bfccd17d22879dc34"}
Jul 06 19:55:32.402148 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:32.402061    2577 generic.go:356] "Generic (PLEG): container finished" podID="2407df6e-673c-4112-a05d-2d677966703b" containerID="0811296893dfd89697cdef93d486d672d00130a16572c643e8bfb731d1e1d1ab" exitCode=0
Jul 06 19:55:32.402148 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:32.402133    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h7nr5" event={"ID":"2407df6e-673c-4112-a05d-2d677966703b","Type":"ContainerDied","Data":"0811296893dfd89697cdef93d486d672d00130a16572c643e8bfb731d1e1d1ab"}
Jul 06 19:55:33.407883 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:33.407850    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h7nr5" event={"ID":"2407df6e-673c-4112-a05d-2d677966703b","Type":"ContainerStarted","Data":"5292c32fd8a9309e9c2d7e5be6717013ee970feeea0d8eb39e86d95f29e1ce9b"}
Jul 06 19:55:33.408271 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:33.407895    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h7nr5" event={"ID":"2407df6e-673c-4112-a05d-2d677966703b","Type":"ContainerStarted","Data":"f14cca044b0eaff671413b36cc78c06f3f6222de5f083b80351433336491cbc1"}
Jul 06 19:55:33.448572 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:33.448524    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h7nr5" podStartSLOduration=2.525377448 podStartE2EDuration="3.448508908s" totalImagesPullingTime="859.456721ms" totalInitContainerRuntime="63.674739ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:33.448263238 +0000 UTC m=+313.759718460" watchObservedRunningTime="2026-07-06 19:55:33.448508908 +0000 UTC m=+313.759964129"
Jul 06 19:55:35.345086 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:35.345054    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7887966bdc-54bwk"
Jul 06 19:55:36.746245 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.746214    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 19:55:36.749722 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.749705    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.757446 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.757423    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hns8n\""
Jul 06 19:55:36.757571 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.757431    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 06 19:55:36.757825 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.757808    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2b8jr63umeae1\""
Jul 06 19:55:36.758248 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.758226    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 06 19:55:36.758248 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.758243    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 06 19:55:36.758414 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.758253    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 06 19:55:36.758414 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.758264    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 06 19:55:36.758414 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.758264    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 06 19:55:36.758414 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.758246    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 06 19:55:36.758619 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.758606    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 06 19:55:36.758703 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.758689    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 06 19:55:36.759394 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.759378    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 06 19:55:36.759872 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.759858    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 06 19:55:36.761120 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.761097    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 06 19:55:36.761651 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.761612    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 06 19:55:36.762199 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.762169    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 06 19:55:36.763652 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.763638    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 06 19:55:36.780440 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.780390    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 19:55:36.818695 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.818669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.818812 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.818699    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.818812 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.818721    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-config\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.818812 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.818745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.818812 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.818792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.818941 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.818835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.818941 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.818853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.818941 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.818868    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.818941 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.818886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.819066 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.818952    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.819066 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.818992    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.819066 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.819015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2844f\" (UniqueName: \"kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-kube-api-access-2844f\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.819066 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.819035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.819066 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.819052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.819245 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.819086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.819245 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.819126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.819245 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.819149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.819245 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.819166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.819245 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.819206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.819245 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.819221    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.920504 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.920504 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920507    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.920755 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.920755 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2844f\" (UniqueName: \"kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-kube-api-access-2844f\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.920755 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.920755 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.920755 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.920755 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.920755 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.920755 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-config\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920906    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920934    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.920978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.921002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921123 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.921024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921705 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.921403    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921797 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.921769    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.921936 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.921913    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.922293 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.922266    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.923112 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.922786    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.924131 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.924109    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.924254 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.924142    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.924459 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.924433    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.924607 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.924584    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.925413 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.924689    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.925413 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.924784    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.925413 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.924928    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.925633 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.925357    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-config\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.925633 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.925357    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.925633 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.925558    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.926245 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.926227    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.926431 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.926415    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.926853 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.926829    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.927279 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.927259    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:36.939865 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:36.939846    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2844f\" (UniqueName: \"kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-kube-api-access-2844f\") pod \"prometheus-k8s-0\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:37.058335 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:37.058252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:55:37.196371 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:37.196236    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 19:55:37.198667 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:55:37.198636    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd452ae58_5477_4ed0_bd9f_900ca01464a8.slice/crio-d55812d38e2a6e817866620f4134fbef15045ee32a6b031875cbca788c54b7c3 WatchSource:0}: Error finding container d55812d38e2a6e817866620f4134fbef15045ee32a6b031875cbca788c54b7c3: Status 404 returned error can't find the container with id d55812d38e2a6e817866620f4134fbef15045ee32a6b031875cbca788c54b7c3
Jul 06 19:55:37.425202 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:37.425147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerStarted","Data":"d55812d38e2a6e817866620f4134fbef15045ee32a6b031875cbca788c54b7c3"}
Jul 06 19:55:38.430008 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:38.429975    2577 generic.go:356] "Generic (PLEG): container finished" podID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerID="3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58" exitCode=0
Jul 06 19:55:38.430412 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:38.430040    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerDied","Data":"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"}
Jul 06 19:55:42.444654 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:42.444614    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerStarted","Data":"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"}
Jul 06 19:55:42.445042 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:42.444693    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerStarted","Data":"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"}
Jul 06 19:55:44.455975 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:44.455940    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerStarted","Data":"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"}
Jul 06 19:55:44.456393 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:44.456018    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerStarted","Data":"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"}
Jul 06 19:55:44.456393 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:44.456058    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerStarted","Data":"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"}
Jul 06 19:55:44.456393 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:44.456103    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerStarted","Data":"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"}
Jul 06 19:55:44.490354 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:44.490278    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.377026915 podStartE2EDuration="8.490255989s" totalImagesPullingTime="6.094246123s" totalInitContainerRuntime="18.982951ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:44.488429422 +0000 UTC m=+324.799884644" watchObservedRunningTime="2026-07-06 19:55:44.490255989 +0000 UTC m=+324.801711210"
Jul 06 19:55:47.058397 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:55:47.058351    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:34.173036 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:34.172979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:56:34.175249 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:34.175220    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b8e220-72ef-4627-9a4e-2e57eb63aa7c-metrics-certs\") pod \"network-metrics-daemon-6dbm7\" (UID: \"17b8e220-72ef-4627-9a4e-2e57eb63aa7c\") " pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:56:34.234907 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:34.234878    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2fwrm\""
Jul 06 19:56:34.242969 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:34.242954    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dbm7"
Jul 06 19:56:34.362522 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:34.362386    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6dbm7"]
Jul 06 19:56:34.366160 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:56:34.366132    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b8e220_72ef_4627_9a4e_2e57eb63aa7c.slice/crio-e42cacb46a1c00e764f7bd18ea2277ea6892e1657704b6f19e9717433a0232a5 WatchSource:0}: Error finding container e42cacb46a1c00e764f7bd18ea2277ea6892e1657704b6f19e9717433a0232a5: Status 404 returned error can't find the container with id e42cacb46a1c00e764f7bd18ea2277ea6892e1657704b6f19e9717433a0232a5
Jul 06 19:56:34.621961 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:34.621928    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dbm7" event={"ID":"17b8e220-72ef-4627-9a4e-2e57eb63aa7c","Type":"ContainerStarted","Data":"e42cacb46a1c00e764f7bd18ea2277ea6892e1657704b6f19e9717433a0232a5"}
Jul 06 19:56:35.626797 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:35.626765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dbm7" event={"ID":"17b8e220-72ef-4627-9a4e-2e57eb63aa7c","Type":"ContainerStarted","Data":"4b0c3db1de245674b4382b3bcddd95c29605c2693ca419adbfa648daba091d2a"}
Jul 06 19:56:35.627201 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:35.626817    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dbm7" event={"ID":"17b8e220-72ef-4627-9a4e-2e57eb63aa7c","Type":"ContainerStarted","Data":"c47e70ae60cfac693d09816d37300809eb874de9e39daf4fde7e2da1d78ab51c"}
Jul 06 19:56:35.644924 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:35.644876    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6dbm7" podStartSLOduration=374.830373792 podStartE2EDuration="6m15.644859749s" totalImagesPullingTime="814.485957ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:35.643608799 +0000 UTC m=+375.955064018" watchObservedRunningTime="2026-07-06 19:56:35.644859749 +0000 UTC m=+375.956314960"
Jul 06 19:56:37.059341 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:37.059305    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:37.078003 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:37.077976    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:37.647664 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:37.647624    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.083394 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.083357    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 19:56:55.083785 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.083652    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="prometheus" containerID="cri-o://9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba" gracePeriod=600
Jul 06 19:56:55.083785 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.083691    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d" gracePeriod=600
Jul 06 19:56:55.083785 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.083707    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="config-reloader" containerID="cri-o://27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380" gracePeriod=600
Jul 06 19:56:55.083785 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.083730    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy-web" containerID="cri-o://d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c" gracePeriod=600
Jul 06 19:56:55.083785 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.083677    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="thanos-sidecar" containerID="cri-o://2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323" gracePeriod=600
Jul 06 19:56:55.083785 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.083658    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy" containerID="cri-o://e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42" gracePeriod=600
Jul 06 19:56:55.329130 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.329107    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.425708 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.425676    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-metrics-client-certs\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.425880 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.425715    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-kube-rbac-proxy\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.425880 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.425740    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-trusted-ca-bundle\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.425880 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.425772    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-db\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.425880 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.425864    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2844f\" (UniqueName: \"kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-kube-api-access-2844f\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.425904    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.425937    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-thanos-prometheus-http-client-file\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.425965    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-tls\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426091 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.425983    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-kubelet-serving-ca-bundle\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426304 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426143    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:56:55.426304 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426250    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-0\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426304 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426282    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-metrics-client-ca\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426448 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426323    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-config\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426448 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426352    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-tls-assets\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426448 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426387    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-grpc-tls\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426448 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426411    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-web-config\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426448 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426436    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-config-out\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426448 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426430    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:56:55.426744 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426528    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-2\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426744 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426559    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-serving-certs-ca-bundle\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426744 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426600    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-1\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426744 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426635    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d452ae58-5477-4ed0-bd9f-900ca01464a8\" (UID: \"d452ae58-5477-4ed0-bd9f-900ca01464a8\") "
Jul 06 19:56:55.426939 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426845    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.426939 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426861    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.427041 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.426947    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:56:55.427420 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.427389    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:56:55.427528 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.427428    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:56:55.427837 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.427810    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:56:55.428357 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.428327    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:56:55.428821 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.428789    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:56:55.429620 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.429582    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:56:55.430019 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.429984    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-kube-api-access-2844f" (OuterVolumeSpecName: "kube-api-access-2844f") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "kube-api-access-2844f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:56:55.430019 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.430006    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:56:55.430136 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.430081    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:56:55.430136 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.430094    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:56:55.430262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.430219    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:56:55.431135 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.431090    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:56:55.431502 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.431474    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:56:55.431673 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.431654    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-config-out" (OuterVolumeSpecName: "config-out") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:56:55.431993 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.431972    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-config" (OuterVolumeSpecName: "config") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:56:55.432078 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.431993    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:56:55.440466 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.440445    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-web-config" (OuterVolumeSpecName: "web-config") pod "d452ae58-5477-4ed0-bd9f-900ca01464a8" (UID: "d452ae58-5477-4ed0-bd9f-900ca01464a8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:56:55.527411 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527383    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527411 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527411    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527421    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527429    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527440    2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-metrics-client-certs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527447    2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-kube-rbac-proxy\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527456    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-db\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527464    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2844f\" (UniqueName: \"kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-kube-api-access-2844f\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527472    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527480    2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527489    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527496    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527504    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d452ae58-5477-4ed0-bd9f-900ca01464a8-configmap-metrics-client-ca\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527513    2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-config\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527520    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d452ae58-5477-4ed0-bd9f-900ca01464a8-tls-assets\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527528    2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-secret-grpc-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527537    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d452ae58-5477-4ed0-bd9f-900ca01464a8-web-config\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.527582 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.527544    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d452ae58-5477-4ed0-bd9f-900ca01464a8-config-out\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 19:56:55.693511 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693429    2577 generic.go:356] "Generic (PLEG): container finished" podID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerID="41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d" exitCode=0
Jul 06 19:56:55.693511 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693454    2577 generic.go:356] "Generic (PLEG): container finished" podID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerID="e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42" exitCode=0
Jul 06 19:56:55.693511 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693461    2577 generic.go:356] "Generic (PLEG): container finished" podID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerID="d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c" exitCode=0
Jul 06 19:56:55.693511 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693467    2577 generic.go:356] "Generic (PLEG): container finished" podID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerID="2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323" exitCode=0
Jul 06 19:56:55.693511 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693472    2577 generic.go:356] "Generic (PLEG): container finished" podID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerID="27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380" exitCode=0
Jul 06 19:56:55.693511 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693476    2577 generic.go:356] "Generic (PLEG): container finished" podID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerID="9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba" exitCode=0
Jul 06 19:56:55.693788 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693510    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.693788 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693512    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerDied","Data":"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"}
Jul 06 19:56:55.693788 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693579    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerDied","Data":"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"}
Jul 06 19:56:55.693788 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerDied","Data":"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"}
Jul 06 19:56:55.693788 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerDied","Data":"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"}
Jul 06 19:56:55.693788 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerDied","Data":"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"}
Jul 06 19:56:55.693788 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693659    2577 scope.go:117] "RemoveContainer" containerID="41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"
Jul 06 19:56:55.693788 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693678    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerDied","Data":"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"}
Jul 06 19:56:55.693788 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.693702    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d452ae58-5477-4ed0-bd9f-900ca01464a8","Type":"ContainerDied","Data":"d55812d38e2a6e817866620f4134fbef15045ee32a6b031875cbca788c54b7c3"}
Jul 06 19:56:55.701122 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.701101    2577 scope.go:117] "RemoveContainer" containerID="e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"
Jul 06 19:56:55.707974 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.707956    2577 scope.go:117] "RemoveContainer" containerID="d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"
Jul 06 19:56:55.713964 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.713945    2577 scope.go:117] "RemoveContainer" containerID="2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"
Jul 06 19:56:55.715482 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.715459    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 19:56:55.720167 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.720149    2577 scope.go:117] "RemoveContainer" containerID="27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"
Jul 06 19:56:55.722161 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.722144    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 19:56:55.726629 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.726614    2577 scope.go:117] "RemoveContainer" containerID="9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"
Jul 06 19:56:55.732936 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.732915    2577 scope.go:117] "RemoveContainer" containerID="3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"
Jul 06 19:56:55.738676 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.738636    2577 scope.go:117] "RemoveContainer" containerID="41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"
Jul 06 19:56:55.738932 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:56:55.738914    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": container with ID starting with 41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d not found: ID does not exist" containerID="41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"
Jul 06 19:56:55.739011 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.738941    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"} err="failed to get container status \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": rpc error: code = NotFound desc = could not find container \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": container with ID starting with 41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d not found: ID does not exist"
Jul 06 19:56:55.739011 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.738958    2577 scope.go:117] "RemoveContainer" containerID="e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"
Jul 06 19:56:55.739334 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:56:55.739313    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": container with ID starting with e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42 not found: ID does not exist" containerID="e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"
Jul 06 19:56:55.739411 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.739343    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"} err="failed to get container status \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": rpc error: code = NotFound desc = could not find container \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": container with ID starting with e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42 not found: ID does not exist"
Jul 06 19:56:55.739411 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.739363    2577 scope.go:117] "RemoveContainer" containerID="d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"
Jul 06 19:56:55.739638 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:56:55.739619    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": container with ID starting with d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c not found: ID does not exist" containerID="d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"
Jul 06 19:56:55.739680 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.739644    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"} err="failed to get container status \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": rpc error: code = NotFound desc = could not find container \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": container with ID starting with d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c not found: ID does not exist"
Jul 06 19:56:55.739680 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.739660    2577 scope.go:117] "RemoveContainer" containerID="2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"
Jul 06 19:56:55.739874 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:56:55.739858    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": container with ID starting with 2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323 not found: ID does not exist" containerID="2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"
Jul 06 19:56:55.739912 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.739880    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"} err="failed to get container status \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": rpc error: code = NotFound desc = could not find container \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": container with ID starting with 2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323 not found: ID does not exist"
Jul 06 19:56:55.739912 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.739895    2577 scope.go:117] "RemoveContainer" containerID="27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"
Jul 06 19:56:55.740118 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:56:55.740103    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": container with ID starting with 27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380 not found: ID does not exist" containerID="27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"
Jul 06 19:56:55.740162 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.740122    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"} err="failed to get container status \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": rpc error: code = NotFound desc = could not find container \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": container with ID starting with 27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380 not found: ID does not exist"
Jul 06 19:56:55.740162 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.740133    2577 scope.go:117] "RemoveContainer" containerID="9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"
Jul 06 19:56:55.740368 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:56:55.740353    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": container with ID starting with 9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba not found: ID does not exist" containerID="9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"
Jul 06 19:56:55.740405 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.740372    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"} err="failed to get container status \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": rpc error: code = NotFound desc = could not find container \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": container with ID starting with 9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba not found: ID does not exist"
Jul 06 19:56:55.740405 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.740386    2577 scope.go:117] "RemoveContainer" containerID="3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"
Jul 06 19:56:55.740590 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:56:55.740576    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": container with ID starting with 3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58 not found: ID does not exist" containerID="3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"
Jul 06 19:56:55.740628 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.740593    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"} err="failed to get container status \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": rpc error: code = NotFound desc = could not find container \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": container with ID starting with 3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58 not found: ID does not exist"
Jul 06 19:56:55.740628 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.740604    2577 scope.go:117] "RemoveContainer" containerID="41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"
Jul 06 19:56:55.740827 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.740802    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"} err="failed to get container status \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": rpc error: code = NotFound desc = could not find container \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": container with ID starting with 41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d not found: ID does not exist"
Jul 06 19:56:55.740894 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.740829    2577 scope.go:117] "RemoveContainer" containerID="e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"
Jul 06 19:56:55.741038 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.741021    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"} err="failed to get container status \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": rpc error: code = NotFound desc = could not find container \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": container with ID starting with e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42 not found: ID does not exist"
Jul 06 19:56:55.741085 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.741039    2577 scope.go:117] "RemoveContainer" containerID="d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"
Jul 06 19:56:55.741262 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.741245    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"} err="failed to get container status \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": rpc error: code = NotFound desc = could not find container \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": container with ID starting with d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c not found: ID does not exist"
Jul 06 19:56:55.741342 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.741264    2577 scope.go:117] "RemoveContainer" containerID="2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"
Jul 06 19:56:55.741490 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.741474    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"} err="failed to get container status \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": rpc error: code = NotFound desc = could not find container \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": container with ID starting with 2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323 not found: ID does not exist"
Jul 06 19:56:55.741539 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.741490    2577 scope.go:117] "RemoveContainer" containerID="27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"
Jul 06 19:56:55.741677 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.741658    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"} err="failed to get container status \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": rpc error: code = NotFound desc = could not find container \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": container with ID starting with 27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380 not found: ID does not exist"
Jul 06 19:56:55.741715 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.741679    2577 scope.go:117] "RemoveContainer" containerID="9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"
Jul 06 19:56:55.741849 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.741830    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"} err="failed to get container status \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": rpc error: code = NotFound desc = could not find container \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": container with ID starting with 9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba not found: ID does not exist"
Jul 06 19:56:55.741913 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.741850    2577 scope.go:117] "RemoveContainer" containerID="3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"
Jul 06 19:56:55.742063 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.742044    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"} err="failed to get container status \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": rpc error: code = NotFound desc = could not find container \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": container with ID starting with 3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58 not found: ID does not exist"
Jul 06 19:56:55.742103 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.742064    2577 scope.go:117] "RemoveContainer" containerID="41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"
Jul 06 19:56:55.742281 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.742260    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"} err="failed to get container status \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": rpc error: code = NotFound desc = could not find container \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": container with ID starting with 41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d not found: ID does not exist"
Jul 06 19:56:55.742281 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.742281    2577 scope.go:117] "RemoveContainer" containerID="e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"
Jul 06 19:56:55.742508 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.742491    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"} err="failed to get container status \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": rpc error: code = NotFound desc = could not find container \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": container with ID starting with e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42 not found: ID does not exist"
Jul 06 19:56:55.742618 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.742511    2577 scope.go:117] "RemoveContainer" containerID="d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"
Jul 06 19:56:55.742736 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.742720    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"} err="failed to get container status \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": rpc error: code = NotFound desc = could not find container \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": container with ID starting with d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c not found: ID does not exist"
Jul 06 19:56:55.742779 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.742737    2577 scope.go:117] "RemoveContainer" containerID="2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"
Jul 06 19:56:55.742905 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.742891    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"} err="failed to get container status \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": rpc error: code = NotFound desc = could not find container \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": container with ID starting with 2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323 not found: ID does not exist"
Jul 06 19:56:55.742945 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.742905    2577 scope.go:117] "RemoveContainer" containerID="27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"
Jul 06 19:56:55.743068 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743054    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"} err="failed to get container status \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": rpc error: code = NotFound desc = could not find container \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": container with ID starting with 27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380 not found: ID does not exist"
Jul 06 19:56:55.743111 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743068    2577 scope.go:117] "RemoveContainer" containerID="9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"
Jul 06 19:56:55.743250 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743233    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"} err="failed to get container status \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": rpc error: code = NotFound desc = could not find container \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": container with ID starting with 9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba not found: ID does not exist"
Jul 06 19:56:55.743319 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743251    2577 scope.go:117] "RemoveContainer" containerID="3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"
Jul 06 19:56:55.743429 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743413    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"} err="failed to get container status \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": rpc error: code = NotFound desc = could not find container \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": container with ID starting with 3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58 not found: ID does not exist"
Jul 06 19:56:55.743475 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743429    2577 scope.go:117] "RemoveContainer" containerID="41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"
Jul 06 19:56:55.743610 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743594    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"} err="failed to get container status \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": rpc error: code = NotFound desc = could not find container \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": container with ID starting with 41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d not found: ID does not exist"
Jul 06 19:56:55.743647 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743611    2577 scope.go:117] "RemoveContainer" containerID="e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"
Jul 06 19:56:55.743791 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743774    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"} err="failed to get container status \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": rpc error: code = NotFound desc = could not find container \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": container with ID starting with e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42 not found: ID does not exist"
Jul 06 19:56:55.743836 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743792    2577 scope.go:117] "RemoveContainer" containerID="d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"
Jul 06 19:56:55.743955 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743941    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"} err="failed to get container status \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": rpc error: code = NotFound desc = could not find container \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": container with ID starting with d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c not found: ID does not exist"
Jul 06 19:56:55.744004 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.743954    2577 scope.go:117] "RemoveContainer" containerID="2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"
Jul 06 19:56:55.744128 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.744110    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"} err="failed to get container status \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": rpc error: code = NotFound desc = could not find container \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": container with ID starting with 2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323 not found: ID does not exist"
Jul 06 19:56:55.744167 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.744130    2577 scope.go:117] "RemoveContainer" containerID="27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"
Jul 06 19:56:55.744375 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.744358    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"} err="failed to get container status \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": rpc error: code = NotFound desc = could not find container \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": container with ID starting with 27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380 not found: ID does not exist"
Jul 06 19:56:55.744375 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.744373    2577 scope.go:117] "RemoveContainer" containerID="9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"
Jul 06 19:56:55.744565 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.744550    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"} err="failed to get container status \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": rpc error: code = NotFound desc = could not find container \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": container with ID starting with 9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba not found: ID does not exist"
Jul 06 19:56:55.744618 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.744564    2577 scope.go:117] "RemoveContainer" containerID="3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"
Jul 06 19:56:55.744750 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.744735    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"} err="failed to get container status \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": rpc error: code = NotFound desc = could not find container \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": container with ID starting with 3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58 not found: ID does not exist"
Jul 06 19:56:55.744786 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.744750    2577 scope.go:117] "RemoveContainer" containerID="41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"
Jul 06 19:56:55.744945 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.744929    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"} err="failed to get container status \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": rpc error: code = NotFound desc = could not find container \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": container with ID starting with 41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d not found: ID does not exist"
Jul 06 19:56:55.744945 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.744945    2577 scope.go:117] "RemoveContainer" containerID="e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"
Jul 06 19:56:55.745165 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.745130    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"} err="failed to get container status \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": rpc error: code = NotFound desc = could not find container \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": container with ID starting with e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42 not found: ID does not exist"
Jul 06 19:56:55.745234 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.745166    2577 scope.go:117] "RemoveContainer" containerID="d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"
Jul 06 19:56:55.745494 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.745471    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"} err="failed to get container status \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": rpc error: code = NotFound desc = could not find container \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": container with ID starting with d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c not found: ID does not exist"
Jul 06 19:56:55.745494 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.745493    2577 scope.go:117] "RemoveContainer" containerID="2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"
Jul 06 19:56:55.745812 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.745791    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"} err="failed to get container status \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": rpc error: code = NotFound desc = could not find container \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": container with ID starting with 2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323 not found: ID does not exist"
Jul 06 19:56:55.745890 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.745813    2577 scope.go:117] "RemoveContainer" containerID="27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"
Jul 06 19:56:55.746060 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.746036    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"} err="failed to get container status \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": rpc error: code = NotFound desc = could not find container \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": container with ID starting with 27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380 not found: ID does not exist"
Jul 06 19:56:55.746060 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.746060    2577 scope.go:117] "RemoveContainer" containerID="9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"
Jul 06 19:56:55.746385 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.746365    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"} err="failed to get container status \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": rpc error: code = NotFound desc = could not find container \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": container with ID starting with 9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba not found: ID does not exist"
Jul 06 19:56:55.746449 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.746399    2577 scope.go:117] "RemoveContainer" containerID="3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"
Jul 06 19:56:55.746643 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.746618    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"} err="failed to get container status \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": rpc error: code = NotFound desc = could not find container \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": container with ID starting with 3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58 not found: ID does not exist"
Jul 06 19:56:55.746643 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.746635    2577 scope.go:117] "RemoveContainer" containerID="41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"
Jul 06 19:56:55.746856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.746835    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d"} err="failed to get container status \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": rpc error: code = NotFound desc = could not find container \"41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d\": container with ID starting with 41386d0d53566f9b0a499e578e9b70a1d2d293338b362a0e297897ff9e6a372d not found: ID does not exist"
Jul 06 19:56:55.746856 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.746855    2577 scope.go:117] "RemoveContainer" containerID="e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"
Jul 06 19:56:55.747038 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747020    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 19:56:55.747090 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747043    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42"} err="failed to get container status \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": rpc error: code = NotFound desc = could not find container \"e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42\": container with ID starting with e7cea6396efcfa0a4e4d4556b54d480fdd36ced6158a533038d27fa40f717a42 not found: ID does not exist"
Jul 06 19:56:55.747090 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747064    2577 scope.go:117] "RemoveContainer" containerID="d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"
Jul 06 19:56:55.747270 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747257    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy"
Jul 06 19:56:55.747354 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747271    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy"
Jul 06 19:56:55.747354 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747281    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy-thanos"
Jul 06 19:56:55.747354 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747289    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy-thanos"
Jul 06 19:56:55.747354 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747289    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c"} err="failed to get container status \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": rpc error: code = NotFound desc = could not find container \"d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c\": container with ID starting with d11d2a41f56454ad8e7437a506fd6582d1986b3b987f3bcf95164aea64f74a0c not found: ID does not exist"
Jul 06 19:56:55.747354 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747312    2577 scope.go:117] "RemoveContainer" containerID="2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"
Jul 06 19:56:55.747354 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747300    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="config-reloader"
Jul 06 19:56:55.747354 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747357    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="config-reloader"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747366    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="thanos-sidecar"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747372    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="thanos-sidecar"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747379    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="prometheus"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747387    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="prometheus"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747397    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="init-config-reloader"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747402    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="init-config-reloader"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747415    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy-web"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747421    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy-web"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747481    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="config-reloader"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747491    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy-thanos"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747498    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="thanos-sidecar"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747504    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy-web"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747509    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="kube-rbac-proxy"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747513    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" containerName="prometheus"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747562    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323"} err="failed to get container status \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": rpc error: code = NotFound desc = could not find container \"2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323\": container with ID starting with 2b0e28e584447fd9860256c2520d453418d28574c49de9ca138ac1075e0b3323 not found: ID does not exist"
Jul 06 19:56:55.747597 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747577    2577 scope.go:117] "RemoveContainer" containerID="27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"
Jul 06 19:56:55.748132 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747791    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380"} err="failed to get container status \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": rpc error: code = NotFound desc = could not find container \"27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380\": container with ID starting with 27a66ab964831acdf89c72312259c48006ceb8b732fd782e9432aa5cacb75380 not found: ID does not exist"
Jul 06 19:56:55.748132 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.747811    2577 scope.go:117] "RemoveContainer" containerID="9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"
Jul 06 19:56:55.748132 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.748008    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba"} err="failed to get container status \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": rpc error: code = NotFound desc = could not find container \"9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba\": container with ID starting with 9c4bc9c8b6d4f58b4b8803e81ada61d15759b2fd0d09bdead5db0c4fbdcaf8ba not found: ID does not exist"
Jul 06 19:56:55.748132 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.748031    2577 scope.go:117] "RemoveContainer" containerID="3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"
Jul 06 19:56:55.748297 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.748247    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58"} err="failed to get container status \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": rpc error: code = NotFound desc = could not find container \"3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58\": container with ID starting with 3809a7a5494fc04bdd6e16ada6898cfbcc8ece717fa2ade00cb63a4feeb34f58 not found: ID does not exist"
Jul 06 19:56:55.752831 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.752816    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.755454 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.755416    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 06 19:56:55.755454 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.755427    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 06 19:56:55.755609 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.755485    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 06 19:56:55.755609 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.755416    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 06 19:56:55.755609 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.755492    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 06 19:56:55.756075 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.756059    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2b8jr63umeae1\""
Jul 06 19:56:55.756394 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.756369    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 06 19:56:55.756476 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.756439    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hns8n\""
Jul 06 19:56:55.756822 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.756808    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 06 19:56:55.756896 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.756872    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 06 19:56:55.757237 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.757071    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 06 19:56:55.757237 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.757072    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 06 19:56:55.757237 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.757118    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 06 19:56:55.757237 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.757128    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 06 19:56:55.757237 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.757072    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 06 19:56:55.759026 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.759009    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 06 19:56:55.762016 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.761999    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 06 19:56:55.772604 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.772583    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 19:56:55.829280 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.829247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57b648be-e24d-47cd-b26e-9c6c65040474-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.829480 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.829285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.829732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.829828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.829875    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.829908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.829939    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.829964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mg26\" (UniqueName: \"kubernetes.io/projected/57b648be-e24d-47cd-b26e-9c6c65040474-kube-api-access-5mg26\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-config\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-web-config\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57b648be-e24d-47cd-b26e-9c6c65040474-config-out\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.833400 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.834064 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830359    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.834064 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.830387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931160 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931160 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931403 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931403 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-web-config\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931403 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57b648be-e24d-47cd-b26e-9c6c65040474-config-out\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931403 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931403 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931662 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931662 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57b648be-e24d-47cd-b26e-9c6c65040474-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931662 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931810 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931859 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931815    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931858    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.931903 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931863    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.932004 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.931945    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.932190 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.932144    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.933888 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.932554    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.933888 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.932595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.933888 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.932664    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mg26\" (UniqueName: \"kubernetes.io/projected/57b648be-e24d-47cd-b26e-9c6c65040474-kube-api-access-5mg26\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.933888 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.932699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.933888 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.932756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.933888 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.932799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-config\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.933888 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.932823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.933888 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.933110    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.933888 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.933673    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.934401 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.933949    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.934401 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.934320    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-web-config\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.934401 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.934351    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.934401 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.934366    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.934590 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.934470    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.934590 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.934485    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57b648be-e24d-47cd-b26e-9c6c65040474-config-out\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.935353 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.935329    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.936368 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.936337    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.936473 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.936455    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.936517 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.936455    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.936517 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.936496    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57b648be-e24d-47cd-b26e-9c6c65040474-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.936883 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.936862    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-config\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.936964 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.936948    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/57b648be-e24d-47cd-b26e-9c6c65040474-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.937592 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.937576    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/57b648be-e24d-47cd-b26e-9c6c65040474-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:55.941716 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:55.941695    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mg26\" (UniqueName: \"kubernetes.io/projected/57b648be-e24d-47cd-b26e-9c6c65040474-kube-api-access-5mg26\") pod \"prometheus-k8s-0\" (UID: \"57b648be-e24d-47cd-b26e-9c6c65040474\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:56.062199 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:56.062088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:56:56.221880 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:56.221781    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 19:56:56.225720 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:56:56.225697    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b648be_e24d_47cd_b26e_9c6c65040474.slice/crio-8915c3c38f5f8025247ffdcd7e42bedc07988cdbf483d007c5b2f8821477f871 WatchSource:0}: Error finding container 8915c3c38f5f8025247ffdcd7e42bedc07988cdbf483d007c5b2f8821477f871: Status 404 returned error can't find the container with id 8915c3c38f5f8025247ffdcd7e42bedc07988cdbf483d007c5b2f8821477f871
Jul 06 19:56:56.337042 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:56.336965    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d452ae58-5477-4ed0-bd9f-900ca01464a8" path="/var/lib/kubelet/pods/d452ae58-5477-4ed0-bd9f-900ca01464a8/volumes"
Jul 06 19:56:56.698592 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:56.698546    2577 generic.go:356] "Generic (PLEG): container finished" podID="57b648be-e24d-47cd-b26e-9c6c65040474" containerID="ff232a96c8e38411e5b037e362eeb06334b8cbd4d5c590be903dee092f9fdc7a" exitCode=0
Jul 06 19:56:56.698592 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:56.698596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57b648be-e24d-47cd-b26e-9c6c65040474","Type":"ContainerDied","Data":"ff232a96c8e38411e5b037e362eeb06334b8cbd4d5c590be903dee092f9fdc7a"}
Jul 06 19:56:56.698772 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:56.698654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57b648be-e24d-47cd-b26e-9c6c65040474","Type":"ContainerStarted","Data":"8915c3c38f5f8025247ffdcd7e42bedc07988cdbf483d007c5b2f8821477f871"}
Jul 06 19:56:57.704791 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:57.704761    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57b648be-e24d-47cd-b26e-9c6c65040474","Type":"ContainerStarted","Data":"41b2922e04cb4807d85becdf2ff6c39cf7b2974bc73ace0042be6f7a78c58770"}
Jul 06 19:56:57.705168 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:57.704819    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57b648be-e24d-47cd-b26e-9c6c65040474","Type":"ContainerStarted","Data":"0f210429f8c5850770f07e371180558cae510c8892a643de99c8e49673326757"}
Jul 06 19:56:57.705168 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:57.704846    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57b648be-e24d-47cd-b26e-9c6c65040474","Type":"ContainerStarted","Data":"ae9e201fa36b5c129e74f3c657653cc1670732e9c674ae01210fc409656bd364"}
Jul 06 19:56:57.705168 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:57.704874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57b648be-e24d-47cd-b26e-9c6c65040474","Type":"ContainerStarted","Data":"c978f6ca3d2b0ac46fa8677d2c41c6c8c257b24670cd554847e7f46eb0af4bf4"}
Jul 06 19:56:57.705168 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:57.704898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57b648be-e24d-47cd-b26e-9c6c65040474","Type":"ContainerStarted","Data":"24b08e928530ac8f342fdb9f249fe64cc4113c85a59cf2bedcca2c7fb882d035"}
Jul 06 19:56:57.705168 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:57.704919    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57b648be-e24d-47cd-b26e-9c6c65040474","Type":"ContainerStarted","Data":"a358ae3081a766062e93d39f1e6f8dbdef89f0b52970ea00e53849c5c305a2f3"}
Jul 06 19:56:57.738449 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:56:57.738396    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.722375449 podStartE2EDuration="2.73837789s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.002441ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:57.736583171 +0000 UTC m=+398.048038391" watchObservedRunningTime="2026-07-06 19:56:57.73837789 +0000 UTC m=+398.049833109"
Jul 06 19:57:01.063213 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:57:01.063168    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:57:56.062831 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:57:56.062735    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:57:56.078141 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:57:56.078111    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:57:56.921314 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:57:56.921288    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 19:59:00.182005 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.181972    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"]
Jul 06 19:59:00.184141 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.184114    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.187566 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.187546    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 19:59:00.187785 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.187766    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 19:59:00.187899 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.187817    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xmgfq\""
Jul 06 19:59:00.187899 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.187826    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 19:59:00.187899 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.187846    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 19:59:00.209445 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.209419    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"]
Jul 06 19:59:00.270615 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.270578    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-apiservice-cert\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.270781 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.270623    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-tmp\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.270781 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.270694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-manifests\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.270781 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.270713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-webhook-cert\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.270781 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.270747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drp9d\" (UniqueName: \"kubernetes.io/projected/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-kube-api-access-drp9d\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.371888 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.371858    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-apiservice-cert\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.372054 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.371898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-tmp\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.372054 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.371952    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-manifests\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.372054 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.371976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-webhook-cert\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.372168 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.372108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drp9d\" (UniqueName: \"kubernetes.io/projected/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-kube-api-access-drp9d\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.372427 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.372395    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-manifests\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.374324 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.374302    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-tmp\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.374420 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.374383    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-apiservice-cert\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.374460 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.374445    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-webhook-cert\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.381814 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.381789    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drp9d\" (UniqueName: \"kubernetes.io/projected/0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de-kube-api-access-drp9d\") pod \"opendatahub-operator-controller-manager-87fbc8964-wjg49\" (UID: \"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de\") " pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.494032 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.493938    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:00.623649 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:00.623563    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"]
Jul 06 19:59:00.628089 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:59:00.628061    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dda2bc0_02e7_4f52_9d9f_08d5ee0b12de.slice/crio-91a15a2c0b016faa75055280ab9948cba77848c9b67a5b05eb0975168574a3af WatchSource:0}: Error finding container 91a15a2c0b016faa75055280ab9948cba77848c9b67a5b05eb0975168574a3af: Status 404 returned error can't find the container with id 91a15a2c0b016faa75055280ab9948cba77848c9b67a5b05eb0975168574a3af
Jul 06 19:59:01.126633 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:01.126593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49" event={"ID":"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de","Type":"ContainerStarted","Data":"91a15a2c0b016faa75055280ab9948cba77848c9b67a5b05eb0975168574a3af"}
Jul 06 19:59:04.142298 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:04.142261    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49" event={"ID":"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de","Type":"ContainerStarted","Data":"f242c853bc5fe2ae027b5084551809a2d3aa837c055cdfe181454f0e8e304464"}
Jul 06 19:59:05.146794 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:05.146757    2577 generic.go:356] "Generic (PLEG): container finished" podID="0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de" containerID="f242c853bc5fe2ae027b5084551809a2d3aa837c055cdfe181454f0e8e304464" exitCode=0
Jul 06 19:59:05.146794 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:05.146800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49" event={"ID":"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de","Type":"ContainerDied","Data":"f242c853bc5fe2ae027b5084551809a2d3aa837c055cdfe181454f0e8e304464"}
Jul 06 19:59:06.152195 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:06.152142    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49" event={"ID":"0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de","Type":"ContainerStarted","Data":"237864bde8d1ff003d0bdf25ae5f1378147c3b71ff1019827cbd3400f263a5ae"}
Jul 06 19:59:06.152623 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:06.152262    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:06.177819 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:06.177771    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49" podStartSLOduration=1.761306262 podStartE2EDuration="6.177750482s" totalImagesPullingTime="3.3561343s" totalInitContainerRuntime="1.06030992s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:59:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:59:06.175563169 +0000 UTC m=+526.487018391" watchObservedRunningTime="2026-07-06 19:59:06.177750482 +0000 UTC m=+526.489205707"
Jul 06 19:59:17.157681 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:17.157649    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-87fbc8964-wjg49"
Jul 06 19:59:28.683627 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.683585    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"]
Jul 06 19:59:28.690291 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.690270    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.693927 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.693897    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 06 19:59:28.693927 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.693897    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 06 19:59:28.694110 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.693934    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jwgfv\""
Jul 06 19:59:28.694110 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.693897    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 06 19:59:28.694110 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.693910    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 06 19:59:28.694110 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.693897    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 06 19:59:28.696093 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.696074    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"]
Jul 06 19:59:28.811950 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.811913    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/50da2f9f-5234-4a6d-a8bb-1466a6437377-manager-config\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.811950 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.811950    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50da2f9f-5234-4a6d-a8bb-1466a6437377-cert\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.812234 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.812016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/50da2f9f-5234-4a6d-a8bb-1466a6437377-metrics-cert\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.812234 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.812053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqhbd\" (UniqueName: \"kubernetes.io/projected/50da2f9f-5234-4a6d-a8bb-1466a6437377-kube-api-access-fqhbd\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.913263 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.913226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/50da2f9f-5234-4a6d-a8bb-1466a6437377-manager-config\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.913263 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.913265    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50da2f9f-5234-4a6d-a8bb-1466a6437377-cert\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.913462 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.913312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/50da2f9f-5234-4a6d-a8bb-1466a6437377-metrics-cert\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.913462 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.913333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqhbd\" (UniqueName: \"kubernetes.io/projected/50da2f9f-5234-4a6d-a8bb-1466a6437377-kube-api-access-fqhbd\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.913855 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.913833    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/50da2f9f-5234-4a6d-a8bb-1466a6437377-manager-config\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.915753 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.915732    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50da2f9f-5234-4a6d-a8bb-1466a6437377-cert\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.915839 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.915769    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/50da2f9f-5234-4a6d-a8bb-1466a6437377-metrics-cert\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.923218 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.923169    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqhbd\" (UniqueName: \"kubernetes.io/projected/50da2f9f-5234-4a6d-a8bb-1466a6437377-kube-api-access-fqhbd\") pod \"lws-controller-manager-65b8985978-jnprq\" (UID: \"50da2f9f-5234-4a6d-a8bb-1466a6437377\") " pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:28.999968 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:28.999880    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:29.133112 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:29.133082    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"]
Jul 06 19:59:29.136039 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:59:29.136016    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50da2f9f_5234_4a6d_a8bb_1466a6437377.slice/crio-f9146a38cbe958362126390e9247ce136b998c18aaee02b4169274a3a1596162 WatchSource:0}: Error finding container f9146a38cbe958362126390e9247ce136b998c18aaee02b4169274a3a1596162: Status 404 returned error can't find the container with id f9146a38cbe958362126390e9247ce136b998c18aaee02b4169274a3a1596162
Jul 06 19:59:29.231557 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:29.231524    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq" event={"ID":"50da2f9f-5234-4a6d-a8bb-1466a6437377","Type":"ContainerStarted","Data":"f9146a38cbe958362126390e9247ce136b998c18aaee02b4169274a3a1596162"}
Jul 06 19:59:30.656376 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.656337    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"]
Jul 06 19:59:30.662306 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.662279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"
Jul 06 19:59:30.665292 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.665235    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 06 19:59:30.665434 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.665247    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jll6r\""
Jul 06 19:59:30.667646 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.667624    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"]
Jul 06 19:59:30.670326 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.669912    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"
Jul 06 19:59:30.671725 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.671685    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"]
Jul 06 19:59:30.672863 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.672744    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 06 19:59:30.672863 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.672752    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dsdd9\""
Jul 06 19:59:30.682985 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.682951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"]
Jul 06 19:59:30.729057 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.729024    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2372363d-da51-48d9-a704-014f6c2684d3-cert\") pod \"kserve-controller-manager-d6f4b65b8-qzc5s\" (UID: \"2372363d-da51-48d9-a704-014f6c2684d3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"
Jul 06 19:59:30.729057 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.729067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsv6t\" (UniqueName: \"kubernetes.io/projected/2372363d-da51-48d9-a704-014f6c2684d3-kube-api-access-zsv6t\") pod \"kserve-controller-manager-d6f4b65b8-qzc5s\" (UID: \"2372363d-da51-48d9-a704-014f6c2684d3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"
Jul 06 19:59:30.830231 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.830196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d724be70-74c7-4ecb-9da7-f3a5ca4c19ab-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bgtgn\" (UID: \"d724be70-74c7-4ecb-9da7-f3a5ca4c19ab\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"
Jul 06 19:59:30.830412 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.830247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vphlq\" (UniqueName: \"kubernetes.io/projected/d724be70-74c7-4ecb-9da7-f3a5ca4c19ab-kube-api-access-vphlq\") pod \"llmisvc-controller-manager-777c6bd7d5-bgtgn\" (UID: \"d724be70-74c7-4ecb-9da7-f3a5ca4c19ab\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"
Jul 06 19:59:30.830412 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.830322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2372363d-da51-48d9-a704-014f6c2684d3-cert\") pod \"kserve-controller-manager-d6f4b65b8-qzc5s\" (UID: \"2372363d-da51-48d9-a704-014f6c2684d3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"
Jul 06 19:59:30.830412 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.830342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsv6t\" (UniqueName: \"kubernetes.io/projected/2372363d-da51-48d9-a704-014f6c2684d3-kube-api-access-zsv6t\") pod \"kserve-controller-manager-d6f4b65b8-qzc5s\" (UID: \"2372363d-da51-48d9-a704-014f6c2684d3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"
Jul 06 19:59:30.830578 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:59:30.830446    2577 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 06 19:59:30.830578 ip-10-0-139-90 kubenswrapper[2577]: E0706 19:59:30.830512    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2372363d-da51-48d9-a704-014f6c2684d3-cert podName:2372363d-da51-48d9-a704-014f6c2684d3 nodeName:}" failed. No retries permitted until 2026-07-06 19:59:31.330490093 +0000 UTC m=+551.641945307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2372363d-da51-48d9-a704-014f6c2684d3-cert") pod "kserve-controller-manager-d6f4b65b8-qzc5s" (UID: "2372363d-da51-48d9-a704-014f6c2684d3") : secret "kserve-webhook-server-cert" not found
Jul 06 19:59:30.874679 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.874651    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsv6t\" (UniqueName: \"kubernetes.io/projected/2372363d-da51-48d9-a704-014f6c2684d3-kube-api-access-zsv6t\") pod \"kserve-controller-manager-d6f4b65b8-qzc5s\" (UID: \"2372363d-da51-48d9-a704-014f6c2684d3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"
Jul 06 19:59:30.931456 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.931097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d724be70-74c7-4ecb-9da7-f3a5ca4c19ab-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bgtgn\" (UID: \"d724be70-74c7-4ecb-9da7-f3a5ca4c19ab\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"
Jul 06 19:59:30.931456 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.931145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vphlq\" (UniqueName: \"kubernetes.io/projected/d724be70-74c7-4ecb-9da7-f3a5ca4c19ab-kube-api-access-vphlq\") pod \"llmisvc-controller-manager-777c6bd7d5-bgtgn\" (UID: \"d724be70-74c7-4ecb-9da7-f3a5ca4c19ab\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"
Jul 06 19:59:30.933794 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.933768    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d724be70-74c7-4ecb-9da7-f3a5ca4c19ab-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bgtgn\" (UID: \"d724be70-74c7-4ecb-9da7-f3a5ca4c19ab\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"
Jul 06 19:59:30.942670 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.942644    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vphlq\" (UniqueName: \"kubernetes.io/projected/d724be70-74c7-4ecb-9da7-f3a5ca4c19ab-kube-api-access-vphlq\") pod \"llmisvc-controller-manager-777c6bd7d5-bgtgn\" (UID: \"d724be70-74c7-4ecb-9da7-f3a5ca4c19ab\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"
Jul 06 19:59:30.984835 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:30.984807    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"
Jul 06 19:59:31.334166 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:31.334076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2372363d-da51-48d9-a704-014f6c2684d3-cert\") pod \"kserve-controller-manager-d6f4b65b8-qzc5s\" (UID: \"2372363d-da51-48d9-a704-014f6c2684d3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"
Jul 06 19:59:31.336486 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:31.336460    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2372363d-da51-48d9-a704-014f6c2684d3-cert\") pod \"kserve-controller-manager-d6f4b65b8-qzc5s\" (UID: \"2372363d-da51-48d9-a704-014f6c2684d3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"
Jul 06 19:59:31.488684 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:31.488652    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"]
Jul 06 19:59:31.490135 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:59:31.490109    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd724be70_74c7_4ecb_9da7_f3a5ca4c19ab.slice/crio-a24b99a173f62756cd6c2b77534a89c55391e87dc471260d1f72d9f88937e481 WatchSource:0}: Error finding container a24b99a173f62756cd6c2b77534a89c55391e87dc471260d1f72d9f88937e481: Status 404 returned error can't find the container with id a24b99a173f62756cd6c2b77534a89c55391e87dc471260d1f72d9f88937e481
Jul 06 19:59:31.577037 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:31.577003    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"
Jul 06 19:59:31.717919 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:31.717887    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"]
Jul 06 19:59:31.720722 ip-10-0-139-90 kubenswrapper[2577]: W0706 19:59:31.720689    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2372363d_da51_48d9_a704_014f6c2684d3.slice/crio-4d96926fb4e10b4beb06ee2565928c526c9a667f5184ad56c646b0638e7d9cc3 WatchSource:0}: Error finding container 4d96926fb4e10b4beb06ee2565928c526c9a667f5184ad56c646b0638e7d9cc3: Status 404 returned error can't find the container with id 4d96926fb4e10b4beb06ee2565928c526c9a667f5184ad56c646b0638e7d9cc3
Jul 06 19:59:32.245509 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:32.245478    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq" event={"ID":"50da2f9f-5234-4a6d-a8bb-1466a6437377","Type":"ContainerStarted","Data":"71b383443b34c202afb0e3aff5e42cdb10956b2185f82c945560642b88235ead"}
Jul 06 19:59:32.245688 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:32.245557    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 19:59:32.246639 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:32.246609    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn" event={"ID":"d724be70-74c7-4ecb-9da7-f3a5ca4c19ab","Type":"ContainerStarted","Data":"a24b99a173f62756cd6c2b77534a89c55391e87dc471260d1f72d9f88937e481"}
Jul 06 19:59:32.247679 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:32.247645    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s" event={"ID":"2372363d-da51-48d9-a704-014f6c2684d3","Type":"ContainerStarted","Data":"4d96926fb4e10b4beb06ee2565928c526c9a667f5184ad56c646b0638e7d9cc3"}
Jul 06 19:59:32.263739 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:32.263696    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq" podStartSLOduration=1.992371074 podStartE2EDuration="4.263680613s" totalImagesPullingTime="2.271309539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:59:32.262841321 +0000 UTC m=+552.574296546" watchObservedRunningTime="2026-07-06 19:59:32.263680613 +0000 UTC m=+552.575135834"
Jul 06 19:59:35.266536 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:35.266500    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s" event={"ID":"2372363d-da51-48d9-a704-014f6c2684d3","Type":"ContainerStarted","Data":"9563dfc2d0727008ad1f268d567af77793b14908c0f960fb487a6ba4de398b53"}
Jul 06 19:59:35.267046 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:35.266586    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"
Jul 06 19:59:35.268040 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:35.268018    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn" event={"ID":"d724be70-74c7-4ecb-9da7-f3a5ca4c19ab","Type":"ContainerStarted","Data":"7b4c6c8c5dcdd4c16f4389d1a9e572e6d619f8c2a55996afd40d90e3b50e6a7a"}
Jul 06 19:59:35.268144 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:35.268128    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"
Jul 06 19:59:35.287543 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:35.287499    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s" podStartSLOduration=2.17247595 podStartE2EDuration="5.287485523s" totalImagesPullingTime="3.115009573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:59:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:59:35.286352891 +0000 UTC m=+555.597808114" watchObservedRunningTime="2026-07-06 19:59:35.287485523 +0000 UTC m=+555.598940788"
Jul 06 19:59:35.308271 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:35.308227    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn" podStartSLOduration=1.95947538 podStartE2EDuration="5.308211028s" totalImagesPullingTime="3.348735648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:59:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:59:35.306704073 +0000 UTC m=+555.618159296" watchObservedRunningTime="2026-07-06 19:59:35.308211028 +0000 UTC m=+555.619666249"
Jul 06 19:59:43.254540 ip-10-0-139-90 kubenswrapper[2577]: I0706 19:59:43.254504    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65b8985978-jnprq"
Jul 06 20:00:06.275207 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:00:06.275157    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bgtgn"
Jul 06 20:00:06.278017 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:00:06.277995    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qzc5s"
Jul 06 20:01:01.153466 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.153430    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"]
Jul 06 20:01:01.155687 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.155666    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:01.158277 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.158250    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 20:01:01.158369 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.158316    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kvfcs\""
Jul 06 20:01:01.158369 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.158329    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 20:01:01.168907 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.168886    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"]
Jul 06 20:01:01.256164 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.256129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8ngq\" (UniqueName: \"kubernetes.io/projected/f19aeb88-c753-42bc-81c6-a628d042cea0-kube-api-access-x8ngq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" (UID: \"f19aeb88-c753-42bc-81c6-a628d042cea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:01.256350 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.256267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f19aeb88-c753-42bc-81c6-a628d042cea0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" (UID: \"f19aeb88-c753-42bc-81c6-a628d042cea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:01.357662 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.357617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f19aeb88-c753-42bc-81c6-a628d042cea0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" (UID: \"f19aeb88-c753-42bc-81c6-a628d042cea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:01.357868 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.357678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8ngq\" (UniqueName: \"kubernetes.io/projected/f19aeb88-c753-42bc-81c6-a628d042cea0-kube-api-access-x8ngq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" (UID: \"f19aeb88-c753-42bc-81c6-a628d042cea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:01.358075 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.358049    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f19aeb88-c753-42bc-81c6-a628d042cea0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" (UID: \"f19aeb88-c753-42bc-81c6-a628d042cea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:01.384241 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.384214    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8ngq\" (UniqueName: \"kubernetes.io/projected/f19aeb88-c753-42bc-81c6-a628d042cea0-kube-api-access-x8ngq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" (UID: \"f19aeb88-c753-42bc-81c6-a628d042cea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:01.465559 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.465469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:01.595519 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.595484    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"]
Jul 06 20:01:01.601325 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:01:01.601295    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19aeb88_c753_42bc_81c6_a628d042cea0.slice/crio-e7ad2c00a24f3674c4a62b3f4279e528fa3996017251209baf83913bdf9a0630 WatchSource:0}: Error finding container e7ad2c00a24f3674c4a62b3f4279e528fa3996017251209baf83913bdf9a0630: Status 404 returned error can't find the container with id e7ad2c00a24f3674c4a62b3f4279e528fa3996017251209baf83913bdf9a0630
Jul 06 20:01:01.604550 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:01.604532    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:01:02.583028 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:02.582979    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz" event={"ID":"f19aeb88-c753-42bc-81c6-a628d042cea0","Type":"ContainerStarted","Data":"e7ad2c00a24f3674c4a62b3f4279e528fa3996017251209baf83913bdf9a0630"}
Jul 06 20:01:07.606301 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:07.606263    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz" event={"ID":"f19aeb88-c753-42bc-81c6-a628d042cea0","Type":"ContainerStarted","Data":"ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041"}
Jul 06 20:01:07.606779 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:07.606382    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:07.627030 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:07.626973    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz" podStartSLOduration=0.835505751 podStartE2EDuration="6.6269514s" totalImagesPullingTime="5.791445649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:01:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:01:07.624851643 +0000 UTC m=+647.936306863" watchObservedRunningTime="2026-07-06 20:01:07.6269514 +0000 UTC m=+647.938406621"
Jul 06 20:01:18.613535 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:18.613506    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:20.002112 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.002082    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"]
Jul 06 20:01:20.005501 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.005478    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:20.018783 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.018759    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"]
Jul 06 20:01:20.113105 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.113073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfcab832-f108-4504-a77b-42762d6376d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mt575\" (UID: \"dfcab832-f108-4504-a77b-42762d6376d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:20.113299 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.113149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kvbb\" (UniqueName: \"kubernetes.io/projected/dfcab832-f108-4504-a77b-42762d6376d8-kube-api-access-7kvbb\") pod \"kuadrant-operator-controller-manager-55467d9f7-mt575\" (UID: \"dfcab832-f108-4504-a77b-42762d6376d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:20.214554 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.214515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfcab832-f108-4504-a77b-42762d6376d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mt575\" (UID: \"dfcab832-f108-4504-a77b-42762d6376d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:20.214722 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.214643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kvbb\" (UniqueName: \"kubernetes.io/projected/dfcab832-f108-4504-a77b-42762d6376d8-kube-api-access-7kvbb\") pod \"kuadrant-operator-controller-manager-55467d9f7-mt575\" (UID: \"dfcab832-f108-4504-a77b-42762d6376d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:20.214888 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.214869    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfcab832-f108-4504-a77b-42762d6376d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mt575\" (UID: \"dfcab832-f108-4504-a77b-42762d6376d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:20.222633 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.222606    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kvbb\" (UniqueName: \"kubernetes.io/projected/dfcab832-f108-4504-a77b-42762d6376d8-kube-api-access-7kvbb\") pod \"kuadrant-operator-controller-manager-55467d9f7-mt575\" (UID: \"dfcab832-f108-4504-a77b-42762d6376d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:20.290813 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.290740    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"]
Jul 06 20:01:20.291011 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.290999    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:20.297163 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.297130    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"]
Jul 06 20:01:20.309604 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.309577    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"]
Jul 06 20:01:20.309806 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.309780    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz" podUID="f19aeb88-c753-42bc-81c6-a628d042cea0" containerName="manager" containerID="cri-o://ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041" gracePeriod=2
Jul 06 20:01:20.318773 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.318744    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"]
Jul 06 20:01:20.323779 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.323760    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"]
Jul 06 20:01:20.323948 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.323879    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:20.324947 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:20.324916    2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f19aeb88-c753-42bc-81c6-a628d042cea0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-hggfz\": pods \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:20.326886 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:20.326852    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" is forbidden: User \"system:node:ip-10-0-139-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-90.ec2.internal' and this object" podUID="f19aeb88-c753-42bc-81c6-a628d042cea0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:20.337932 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.337907    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"]
Jul 06 20:01:20.367117 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.367089    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv"]
Jul 06 20:01:20.368516 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.368488    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f19aeb88-c753-42bc-81c6-a628d042cea0" containerName="manager"
Jul 06 20:01:20.368516 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.368517    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f19aeb88-c753-42bc-81c6-a628d042cea0" containerName="manager"
Jul 06 20:01:20.368754 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.368735    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f19aeb88-c753-42bc-81c6-a628d042cea0" containerName="manager"
Jul 06 20:01:20.374202 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.374162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv"
Jul 06 20:01:20.377470 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.377449    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-wbqwq\""
Jul 06 20:01:20.384919 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.384890    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv"]
Jul 06 20:01:20.416080 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.416054    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4brfs\" (UniqueName: \"kubernetes.io/projected/9b52eeca-a79c-4142-a37d-d398a0ec911e-kube-api-access-4brfs\") pod \"kuadrant-operator-controller-manager-55467d9f7-flcpm\" (UID: \"9b52eeca-a79c-4142-a37d-d398a0ec911e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:20.416201 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.416128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b52eeca-a79c-4142-a37d-d398a0ec911e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-flcpm\" (UID: \"9b52eeca-a79c-4142-a37d-d398a0ec911e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:20.517377 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.517344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4brfs\" (UniqueName: \"kubernetes.io/projected/9b52eeca-a79c-4142-a37d-d398a0ec911e-kube-api-access-4brfs\") pod \"kuadrant-operator-controller-manager-55467d9f7-flcpm\" (UID: \"9b52eeca-a79c-4142-a37d-d398a0ec911e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:20.517546 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.517389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr5bd\" (UniqueName: \"kubernetes.io/projected/ff0ce936-bf25-4608-bdae-ee23d8bd9b2a-kube-api-access-qr5bd\") pod \"limitador-operator-controller-manager-859d7b78bd-lztrv\" (UID: \"ff0ce936-bf25-4608-bdae-ee23d8bd9b2a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv"
Jul 06 20:01:20.517546 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.517425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b52eeca-a79c-4142-a37d-d398a0ec911e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-flcpm\" (UID: \"9b52eeca-a79c-4142-a37d-d398a0ec911e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:20.517816 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.517793    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b52eeca-a79c-4142-a37d-d398a0ec911e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-flcpm\" (UID: \"9b52eeca-a79c-4142-a37d-d398a0ec911e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:20.536614 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.536586    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4brfs\" (UniqueName: \"kubernetes.io/projected/9b52eeca-a79c-4142-a37d-d398a0ec911e-kube-api-access-4brfs\") pod \"kuadrant-operator-controller-manager-55467d9f7-flcpm\" (UID: \"9b52eeca-a79c-4142-a37d-d398a0ec911e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:20.540914 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.540867    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:20.543090 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:20.543063    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" is forbidden: User \"system:node:ip-10-0-139-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-90.ec2.internal' and this object" podUID="f19aeb88-c753-42bc-81c6-a628d042cea0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:20.617937 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.617902    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8ngq\" (UniqueName: \"kubernetes.io/projected/f19aeb88-c753-42bc-81c6-a628d042cea0-kube-api-access-x8ngq\") pod \"f19aeb88-c753-42bc-81c6-a628d042cea0\" (UID: \"f19aeb88-c753-42bc-81c6-a628d042cea0\") "
Jul 06 20:01:20.618120 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.617978    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f19aeb88-c753-42bc-81c6-a628d042cea0-extensions-socket-volume\") pod \"f19aeb88-c753-42bc-81c6-a628d042cea0\" (UID: \"f19aeb88-c753-42bc-81c6-a628d042cea0\") "
Jul 06 20:01:20.618120 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.618048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr5bd\" (UniqueName: \"kubernetes.io/projected/ff0ce936-bf25-4608-bdae-ee23d8bd9b2a-kube-api-access-qr5bd\") pod \"limitador-operator-controller-manager-859d7b78bd-lztrv\" (UID: \"ff0ce936-bf25-4608-bdae-ee23d8bd9b2a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv"
Jul 06 20:01:20.618429 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.618402    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19aeb88-c753-42bc-81c6-a628d042cea0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f19aeb88-c753-42bc-81c6-a628d042cea0" (UID: "f19aeb88-c753-42bc-81c6-a628d042cea0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:01:20.620109 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.620087    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f19aeb88-c753-42bc-81c6-a628d042cea0-kube-api-access-x8ngq" (OuterVolumeSpecName: "kube-api-access-x8ngq") pod "f19aeb88-c753-42bc-81c6-a628d042cea0" (UID: "f19aeb88-c753-42bc-81c6-a628d042cea0"). InnerVolumeSpecName "kube-api-access-x8ngq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:01:20.628349 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.628329    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr5bd\" (UniqueName: \"kubernetes.io/projected/ff0ce936-bf25-4608-bdae-ee23d8bd9b2a-kube-api-access-qr5bd\") pod \"limitador-operator-controller-manager-859d7b78bd-lztrv\" (UID: \"ff0ce936-bf25-4608-bdae-ee23d8bd9b2a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv"
Jul 06 20:01:20.658695 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.658666    2577 generic.go:356] "Generic (PLEG): container finished" podID="f19aeb88-c753-42bc-81c6-a628d042cea0" containerID="ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041" exitCode=0
Jul 06 20:01:20.658828 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.658709    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:20.658828 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.658777    2577 scope.go:117] "RemoveContainer" containerID="ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041"
Jul 06 20:01:20.661321 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:20.661297    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" is forbidden: User \"system:node:ip-10-0-139-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-90.ec2.internal' and this object" podUID="f19aeb88-c753-42bc-81c6-a628d042cea0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:20.667023 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.667006    2577 scope.go:117] "RemoveContainer" containerID="ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041"
Jul 06 20:01:20.667289 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:20.667263    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041\": container with ID starting with ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041 not found: ID does not exist" containerID="ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041"
Jul 06 20:01:20.667358 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.667295    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041"} err="failed to get container status \"ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041\": rpc error: code = NotFound desc = could not find container \"ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041\": container with ID starting with ef419cf62d28bcefdd978a2af2ba11afcc85ae157570085c7ba7a748194ff041 not found: ID does not exist"
Jul 06 20:01:20.669030 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:20.669010    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" is forbidden: User \"system:node:ip-10-0-139-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-90.ec2.internal' and this object" podUID="f19aeb88-c753-42bc-81c6-a628d042cea0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:20.697735 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.697711    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:20.703845 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.703830    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv"
Jul 06 20:01:20.719087 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.719064    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8ngq\" (UniqueName: \"kubernetes.io/projected/f19aeb88-c753-42bc-81c6-a628d042cea0-kube-api-access-x8ngq\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:01:20.719087 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.719089    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f19aeb88-c753-42bc-81c6-a628d042cea0-extensions-socket-volume\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:01:20.834047 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.834001    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"]
Jul 06 20:01:20.837107 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:01:20.837076    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b52eeca_a79c_4142_a37d_d398a0ec911e.slice/crio-eb66f38bb08be599485be5a0304039a7cbe12ababd50231e1f8f3fe79fa24662 WatchSource:0}: Error finding container eb66f38bb08be599485be5a0304039a7cbe12ababd50231e1f8f3fe79fa24662: Status 404 returned error can't find the container with id eb66f38bb08be599485be5a0304039a7cbe12ababd50231e1f8f3fe79fa24662
Jul 06 20:01:20.850298 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:01:20.850273    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0ce936_bf25_4608_bdae_ee23d8bd9b2a.slice/crio-941115d2635a1fd9862d008eec2843c8093b13009b7a7590c35d2271a33b1562 WatchSource:0}: Error finding container 941115d2635a1fd9862d008eec2843c8093b13009b7a7590c35d2271a33b1562: Status 404 returned error can't find the container with id 941115d2635a1fd9862d008eec2843c8093b13009b7a7590c35d2271a33b1562
Jul 06 20:01:20.859327 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:20.859301    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv"]
Jul 06 20:01:21.664664 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:21.664628    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv" event={"ID":"ff0ce936-bf25-4608-bdae-ee23d8bd9b2a","Type":"ContainerStarted","Data":"941115d2635a1fd9862d008eec2843c8093b13009b7a7590c35d2271a33b1562"}
Jul 06 20:01:21.667137 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:21.667111    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm" event={"ID":"9b52eeca-a79c-4142-a37d-d398a0ec911e","Type":"ContainerStarted","Data":"7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899"}
Jul 06 20:01:21.667290 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:21.667161    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm" event={"ID":"9b52eeca-a79c-4142-a37d-d398a0ec911e","Type":"ContainerStarted","Data":"eb66f38bb08be599485be5a0304039a7cbe12ababd50231e1f8f3fe79fa24662"}
Jul 06 20:01:21.667361 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:21.667325    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:21.669772 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:21.669748    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hggfz\" is forbidden: User \"system:node:ip-10-0-139-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-90.ec2.internal' and this object" podUID="f19aeb88-c753-42bc-81c6-a628d042cea0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hggfz"
Jul 06 20:01:21.691368 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:21.691313    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm" podStartSLOduration=1.691295972 podStartE2EDuration="1.691295972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:01:21.689510624 +0000 UTC m=+662.000965857" watchObservedRunningTime="2026-07-06 20:01:21.691295972 +0000 UTC m=+662.002751193"
Jul 06 20:01:22.339417 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:22.339387    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f19aeb88-c753-42bc-81c6-a628d042cea0" path="/var/lib/kubelet/pods/f19aeb88-c753-42bc-81c6-a628d042cea0/volumes"
Jul 06 20:01:22.837025 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:22.836992    2577 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-mt575_kuadrant-system_dfcab832-f108-4504-a77b-42762d6376d8_0(d4fea5b8c03fe71e5c50136bba372e6c272b38d46e71e34b8d1be3a01db89e86): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-mt575 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 06 20:01:22.837299 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:22.837050    2577 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-mt575_kuadrant-system_dfcab832-f108-4504-a77b-42762d6376d8_0(d4fea5b8c03fe71e5c50136bba372e6c272b38d46e71e34b8d1be3a01db89e86): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-mt575 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:23.676971 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:23.676941    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:23.676971 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:23.676949    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv" event={"ID":"ff0ce936-bf25-4608-bdae-ee23d8bd9b2a","Type":"ContainerStarted","Data":"46d250a2e47263257c2fbf6b80b888a707381fd45b71bc28af00bc514525cffe"}
Jul 06 20:01:23.677215 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:23.677048    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv"
Jul 06 20:01:23.683366 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:23.683349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:23.695518 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:23.695474    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv" podStartSLOduration=1.7321716870000001 podStartE2EDuration="3.695460316s" totalImagesPullingTime="1.963288629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:01:23.693975737 +0000 UTC m=+664.005430954" watchObservedRunningTime="2026-07-06 20:01:23.695460316 +0000 UTC m=+664.006915536"
Jul 06 20:01:23.695981 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:23.695960    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mt575\" is forbidden: User \"system:node:ip-10-0-139-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-90.ec2.internal' and this object" podUID="dfcab832-f108-4504-a77b-42762d6376d8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:23.697837 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:23.697815    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mt575\" is forbidden: User \"system:node:ip-10-0-139-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-90.ec2.internal' and this object" podUID="dfcab832-f108-4504-a77b-42762d6376d8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:23.743081 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:23.743046    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfcab832-f108-4504-a77b-42762d6376d8-extensions-socket-volume\") pod \"dfcab832-f108-4504-a77b-42762d6376d8\" (UID: \"dfcab832-f108-4504-a77b-42762d6376d8\") "
Jul 06 20:01:23.743256 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:23.743156    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kvbb\" (UniqueName: \"kubernetes.io/projected/dfcab832-f108-4504-a77b-42762d6376d8-kube-api-access-7kvbb\") pod \"dfcab832-f108-4504-a77b-42762d6376d8\" (UID: \"dfcab832-f108-4504-a77b-42762d6376d8\") "
Jul 06 20:01:23.743470 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:23.743443    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfcab832-f108-4504-a77b-42762d6376d8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dfcab832-f108-4504-a77b-42762d6376d8" (UID: "dfcab832-f108-4504-a77b-42762d6376d8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:01:23.745227 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:23.745201    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfcab832-f108-4504-a77b-42762d6376d8-kube-api-access-7kvbb" (OuterVolumeSpecName: "kube-api-access-7kvbb") pod "dfcab832-f108-4504-a77b-42762d6376d8" (UID: "dfcab832-f108-4504-a77b-42762d6376d8"). InnerVolumeSpecName "kube-api-access-7kvbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:01:23.844712 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:23.844677    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfcab832-f108-4504-a77b-42762d6376d8-extensions-socket-volume\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:01:23.844712 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:23.844704    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kvbb\" (UniqueName: \"kubernetes.io/projected/dfcab832-f108-4504-a77b-42762d6376d8-kube-api-access-7kvbb\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:01:24.337363 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:24.337331    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfcab832-f108-4504-a77b-42762d6376d8" path="/var/lib/kubelet/pods/dfcab832-f108-4504-a77b-42762d6376d8/volumes"
Jul 06 20:01:24.680905 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:24.680872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:24.683798 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:24.683770    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mt575\" is forbidden: User \"system:node:ip-10-0-139-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-90.ec2.internal' and this object" podUID="dfcab832-f108-4504-a77b-42762d6376d8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:24.686080 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:24.686053    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mt575\" is forbidden: User \"system:node:ip-10-0-139-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-90.ec2.internal' and this object" podUID="dfcab832-f108-4504-a77b-42762d6376d8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mt575"
Jul 06 20:01:32.674584 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:32.674551    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:33.128341 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.128309    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"]
Jul 06 20:01:33.134644 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.134624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:01:33.146216 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.146191    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"]
Jul 06 20:01:33.230783 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.230691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtrgs\" (UniqueName: \"kubernetes.io/projected/e479ef2a-21df-490c-b995-bb3aa448a710-kube-api-access-gtrgs\") pod \"kuadrant-operator-controller-manager-86878c769f-6llx5\" (UID: \"e479ef2a-21df-490c-b995-bb3aa448a710\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:01:33.230783 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.230730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e479ef2a-21df-490c-b995-bb3aa448a710-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6llx5\" (UID: \"e479ef2a-21df-490c-b995-bb3aa448a710\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:01:33.332025 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.331988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtrgs\" (UniqueName: \"kubernetes.io/projected/e479ef2a-21df-490c-b995-bb3aa448a710-kube-api-access-gtrgs\") pod \"kuadrant-operator-controller-manager-86878c769f-6llx5\" (UID: \"e479ef2a-21df-490c-b995-bb3aa448a710\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:01:33.332025 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.332027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e479ef2a-21df-490c-b995-bb3aa448a710-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6llx5\" (UID: \"e479ef2a-21df-490c-b995-bb3aa448a710\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:01:33.332393 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.332374    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e479ef2a-21df-490c-b995-bb3aa448a710-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6llx5\" (UID: \"e479ef2a-21df-490c-b995-bb3aa448a710\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:01:33.342506 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.342485    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtrgs\" (UniqueName: \"kubernetes.io/projected/e479ef2a-21df-490c-b995-bb3aa448a710-kube-api-access-gtrgs\") pod \"kuadrant-operator-controller-manager-86878c769f-6llx5\" (UID: \"e479ef2a-21df-490c-b995-bb3aa448a710\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:01:33.443642 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.443609    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:01:33.575361 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.575330    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"]
Jul 06 20:01:33.578329 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:01:33.578300    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode479ef2a_21df_490c_b995_bb3aa448a710.slice/crio-8f506e7fe07921bf5e004b4efd224218d7cb8384d35de56f1d2f0f96af1b8b14 WatchSource:0}: Error finding container 8f506e7fe07921bf5e004b4efd224218d7cb8384d35de56f1d2f0f96af1b8b14: Status 404 returned error can't find the container with id 8f506e7fe07921bf5e004b4efd224218d7cb8384d35de56f1d2f0f96af1b8b14
Jul 06 20:01:33.726517 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.726483    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5" event={"ID":"e479ef2a-21df-490c-b995-bb3aa448a710","Type":"ContainerStarted","Data":"072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649"}
Jul 06 20:01:33.726879 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.726539    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5" event={"ID":"e479ef2a-21df-490c-b995-bb3aa448a710","Type":"ContainerStarted","Data":"8f506e7fe07921bf5e004b4efd224218d7cb8384d35de56f1d2f0f96af1b8b14"}
Jul 06 20:01:33.726879 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.726579    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:01:33.746479 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:33.746379    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5" podStartSLOduration=0.746362993 podStartE2EDuration="746.362993ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:01:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:01:33.746006635 +0000 UTC m=+674.057461858" watchObservedRunningTime="2026-07-06 20:01:33.746362993 +0000 UTC m=+674.057818208"
Jul 06 20:01:34.683758 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:34.683729    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lztrv"
Jul 06 20:01:44.733485 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:44.733451    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:01:44.787908 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:44.787879    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"]
Jul 06 20:01:44.788108 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:44.788084    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm" podUID="9b52eeca-a79c-4142-a37d-d398a0ec911e" containerName="manager" containerID="cri-o://7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899" gracePeriod=10
Jul 06 20:01:45.035110 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.035087    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:45.127911 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.127878    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4brfs\" (UniqueName: \"kubernetes.io/projected/9b52eeca-a79c-4142-a37d-d398a0ec911e-kube-api-access-4brfs\") pod \"9b52eeca-a79c-4142-a37d-d398a0ec911e\" (UID: \"9b52eeca-a79c-4142-a37d-d398a0ec911e\") "
Jul 06 20:01:45.128096 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.127925    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b52eeca-a79c-4142-a37d-d398a0ec911e-extensions-socket-volume\") pod \"9b52eeca-a79c-4142-a37d-d398a0ec911e\" (UID: \"9b52eeca-a79c-4142-a37d-d398a0ec911e\") "
Jul 06 20:01:45.128376 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.128346    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b52eeca-a79c-4142-a37d-d398a0ec911e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9b52eeca-a79c-4142-a37d-d398a0ec911e" (UID: "9b52eeca-a79c-4142-a37d-d398a0ec911e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:01:45.129897 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.129880    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b52eeca-a79c-4142-a37d-d398a0ec911e-kube-api-access-4brfs" (OuterVolumeSpecName: "kube-api-access-4brfs") pod "9b52eeca-a79c-4142-a37d-d398a0ec911e" (UID: "9b52eeca-a79c-4142-a37d-d398a0ec911e"). InnerVolumeSpecName "kube-api-access-4brfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:01:45.229111 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.229075    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4brfs\" (UniqueName: \"kubernetes.io/projected/9b52eeca-a79c-4142-a37d-d398a0ec911e-kube-api-access-4brfs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:01:45.229111 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.229106    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b52eeca-a79c-4142-a37d-d398a0ec911e-extensions-socket-volume\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:01:45.778067 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.778031    2577 generic.go:356] "Generic (PLEG): container finished" podID="9b52eeca-a79c-4142-a37d-d398a0ec911e" containerID="7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899" exitCode=0
Jul 06 20:01:45.778067 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.778071    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"
Jul 06 20:01:45.778555 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.778118    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm" event={"ID":"9b52eeca-a79c-4142-a37d-d398a0ec911e","Type":"ContainerDied","Data":"7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899"}
Jul 06 20:01:45.778555 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.778164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm" event={"ID":"9b52eeca-a79c-4142-a37d-d398a0ec911e","Type":"ContainerDied","Data":"eb66f38bb08be599485be5a0304039a7cbe12ababd50231e1f8f3fe79fa24662"}
Jul 06 20:01:45.778555 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.778210    2577 scope.go:117] "RemoveContainer" containerID="7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899"
Jul 06 20:01:45.786395 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.786378    2577 scope.go:117] "RemoveContainer" containerID="7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899"
Jul 06 20:01:45.786647 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:01:45.786630    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899\": container with ID starting with 7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899 not found: ID does not exist" containerID="7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899"
Jul 06 20:01:45.786693 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.786657    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899"} err="failed to get container status \"7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899\": rpc error: code = NotFound desc = could not find container \"7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899\": container with ID starting with 7ce43de22e31904d9f010ab1db43ad4feb0e8da81b4b24ff72a2fadd7a17d899 not found: ID does not exist"
Jul 06 20:01:45.815411 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.815389    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"]
Jul 06 20:01:45.822838 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:45.822785    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-flcpm"]
Jul 06 20:01:46.337060 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:01:46.337023    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b52eeca-a79c-4142-a37d-d398a0ec911e" path="/var/lib/kubelet/pods/9b52eeca-a79c-4142-a37d-d398a0ec911e/volumes"
Jul 06 20:03:23.067107 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.067069    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-xwd5l"]
Jul 06 20:03:23.067681 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.067508    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b52eeca-a79c-4142-a37d-d398a0ec911e" containerName="manager"
Jul 06 20:03:23.067681 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.067526    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b52eeca-a79c-4142-a37d-d398a0ec911e" containerName="manager"
Jul 06 20:03:23.067681 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.067613    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b52eeca-a79c-4142-a37d-d398a0ec911e" containerName="manager"
Jul 06 20:03:23.070433 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.070414    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l"
Jul 06 20:03:23.072702 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.072681    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 20:03:23.072859 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.072830    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 06 20:03:23.072960 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.072943    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-mhhmp\""
Jul 06 20:03:23.073877 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.073859    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 20:03:23.080224 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.080195    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-xwd5l"]
Jul 06 20:03:23.167031 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.166993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v44gx\" (UniqueName: \"kubernetes.io/projected/b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5-kube-api-access-v44gx\") pod \"payload-pre-processing-7964457696-xwd5l\" (UID: \"b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5\") " pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l"
Jul 06 20:03:23.167226 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.167066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5-plugins-config-volume\") pod \"payload-pre-processing-7964457696-xwd5l\" (UID: \"b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5\") " pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l"
Jul 06 20:03:23.267808 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.267769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5-plugins-config-volume\") pod \"payload-pre-processing-7964457696-xwd5l\" (UID: \"b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5\") " pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l"
Jul 06 20:03:23.267965 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.267828    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v44gx\" (UniqueName: \"kubernetes.io/projected/b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5-kube-api-access-v44gx\") pod \"payload-pre-processing-7964457696-xwd5l\" (UID: \"b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5\") " pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l"
Jul 06 20:03:23.268456 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.268438    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5-plugins-config-volume\") pod \"payload-pre-processing-7964457696-xwd5l\" (UID: \"b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5\") " pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l"
Jul 06 20:03:23.280048 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.280026    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v44gx\" (UniqueName: \"kubernetes.io/projected/b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5-kube-api-access-v44gx\") pod \"payload-pre-processing-7964457696-xwd5l\" (UID: \"b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5\") " pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l"
Jul 06 20:03:23.381321 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.381294    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l"
Jul 06 20:03:23.508335 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:23.508302    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-xwd5l"]
Jul 06 20:03:23.511784 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:03:23.511759    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36cb15e_1d70_4b8a_8523_6fd9b5ab3dd5.slice/crio-1a6e4e8b448d8356ead136a670e036bb2e84f383befd8529158f5ff1d4142f55 WatchSource:0}: Error finding container 1a6e4e8b448d8356ead136a670e036bb2e84f383befd8529158f5ff1d4142f55: Status 404 returned error can't find the container with id 1a6e4e8b448d8356ead136a670e036bb2e84f383befd8529158f5ff1d4142f55
Jul 06 20:03:24.178569 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:24.178539    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l" event={"ID":"b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5","Type":"ContainerStarted","Data":"1a6e4e8b448d8356ead136a670e036bb2e84f383befd8529158f5ff1d4142f55"}
Jul 06 20:03:26.189720 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:26.189685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l" event={"ID":"b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5","Type":"ContainerStarted","Data":"c229427ba95f3e01f1aa23eec1349fd877652d275cb1e786c43cb1baa089c56c"}
Jul 06 20:03:26.190111 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:26.189795    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l"
Jul 06 20:03:26.206036 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:26.205994    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l" podStartSLOduration=1.593148168 podStartE2EDuration="3.205980799s" totalImagesPullingTime="1.612832631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:03:26.20547163 +0000 UTC m=+786.516926851" watchObservedRunningTime="2026-07-06 20:03:26.205980799 +0000 UTC m=+786.517436019"
Jul 06 20:03:37.196200 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:03:37.196131    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-xwd5l"
Jul 06 20:04:20.670021 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.669990    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"]
Jul 06 20:04:20.672729 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.672707    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.676776 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.676756    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 20:04:20.676894 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.676806    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-75vhw\""
Jul 06 20:04:20.676894 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.676762    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 06 20:04:20.676894 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.676756    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 20:04:20.690137 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.690113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"]
Jul 06 20:04:20.744271 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.744229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.744441 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.744313    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb318378-0c93-4df6-8254-1e80b227445b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.744441 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.744353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.744441 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.744376    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzkg9\" (UniqueName: \"kubernetes.io/projected/eb318378-0c93-4df6-8254-1e80b227445b-kube-api-access-zzkg9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.744441 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.744428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.744591 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.744486    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.845063 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.845025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb318378-0c93-4df6-8254-1e80b227445b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.845288 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.845071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.845288 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.845097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzkg9\" (UniqueName: \"kubernetes.io/projected/eb318378-0c93-4df6-8254-1e80b227445b-kube-api-access-zzkg9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.845288 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.845150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.845288 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.845206    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.845288 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.845239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.845564 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.845542    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.845612 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.845578    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.845612 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.845592    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.847464 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.847443    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb318378-0c93-4df6-8254-1e80b227445b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.847613 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.847597    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb318378-0c93-4df6-8254-1e80b227445b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.853067 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.853046    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzkg9\" (UniqueName: \"kubernetes.io/projected/eb318378-0c93-4df6-8254-1e80b227445b-kube-api-access-zzkg9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98crx\" (UID: \"eb318378-0c93-4df6-8254-1e80b227445b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:20.982333 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:20.982246    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:21.110506 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:21.110365    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"]
Jul 06 20:04:21.112331 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:04:21.112296    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb318378_0c93_4df6_8254_1e80b227445b.slice/crio-51746b7a6b6168d2b67cbaa4457a1e5ecf9e9ead1292e91ce67145f35d05a847 WatchSource:0}: Error finding container 51746b7a6b6168d2b67cbaa4457a1e5ecf9e9ead1292e91ce67145f35d05a847: Status 404 returned error can't find the container with id 51746b7a6b6168d2b67cbaa4457a1e5ecf9e9ead1292e91ce67145f35d05a847
Jul 06 20:04:21.423273 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:21.423237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx" event={"ID":"eb318378-0c93-4df6-8254-1e80b227445b","Type":"ContainerStarted","Data":"51746b7a6b6168d2b67cbaa4457a1e5ecf9e9ead1292e91ce67145f35d05a847"}
Jul 06 20:04:22.429961 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:22.429870    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx" event={"ID":"eb318378-0c93-4df6-8254-1e80b227445b","Type":"ContainerStarted","Data":"5501a1b7cef0b9e40ec983e48b9790f72b6dfdb52bad157bc8ea216e498f54c1"}
Jul 06 20:04:22.450540 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:22.450472    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx" podStartSLOduration=1.440944932 podStartE2EDuration="2.450451664s" totalImagesPullingTime="1.009506732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:04:22.446878491 +0000 UTC m=+842.758333710" watchObservedRunningTime="2026-07-06 20:04:22.450451664 +0000 UTC m=+842.761906884"
Jul 06 20:04:28.058091 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.058060    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"]
Jul 06 20:04:28.061314 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.061293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.063909 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.063887    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 06 20:04:28.072551 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.072524    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"]
Jul 06 20:04:28.105661 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.105628    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.105812 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.105668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.105812 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.105684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302d0400-1481-46c3-8ce6-b0ea575ba77d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.105812 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.105704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s5vk\" (UniqueName: \"kubernetes.io/projected/302d0400-1481-46c3-8ce6-b0ea575ba77d-kube-api-access-4s5vk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.105812 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.105791    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.105941 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.105824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.207095 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.207004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.207305 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.207139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.207305 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.207226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.207305 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.207270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.207305 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.207295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302d0400-1481-46c3-8ce6-b0ea575ba77d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.207507 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.207329    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s5vk\" (UniqueName: \"kubernetes.io/projected/302d0400-1481-46c3-8ce6-b0ea575ba77d-kube-api-access-4s5vk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.207507 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.207448    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.207787 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.207758    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.207912 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.207890    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.209820 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.209796    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302d0400-1481-46c3-8ce6-b0ea575ba77d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.209929 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.209796    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302d0400-1481-46c3-8ce6-b0ea575ba77d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.215001 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.214982    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s5vk\" (UniqueName: \"kubernetes.io/projected/302d0400-1481-46c3-8ce6-b0ea575ba77d-kube-api-access-4s5vk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d\" (UID: \"302d0400-1481-46c3-8ce6-b0ea575ba77d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.371400 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.371366    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:28.501018 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:28.500981    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"]
Jul 06 20:04:28.504812 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:04:28.504785    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod302d0400_1481_46c3_8ce6_b0ea575ba77d.slice/crio-9162a313868904a2f0f55fad88cadc20588d2c1cf50215ef1c58ad369e8e441e WatchSource:0}: Error finding container 9162a313868904a2f0f55fad88cadc20588d2c1cf50215ef1c58ad369e8e441e: Status 404 returned error can't find the container with id 9162a313868904a2f0f55fad88cadc20588d2c1cf50215ef1c58ad369e8e441e
Jul 06 20:04:29.461706 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:29.461670    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d" event={"ID":"302d0400-1481-46c3-8ce6-b0ea575ba77d","Type":"ContainerStarted","Data":"4839721eafc7ce3de6a1867f4b9fbdf507b1cffa991f29cc30b016c23a98ae9c"}
Jul 06 20:04:29.462083 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:29.461721    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d" event={"ID":"302d0400-1481-46c3-8ce6-b0ea575ba77d","Type":"ContainerStarted","Data":"9162a313868904a2f0f55fad88cadc20588d2c1cf50215ef1c58ad369e8e441e"}
Jul 06 20:04:29.483352 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:29.483302    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d" podStartSLOduration=1.271486382 podStartE2EDuration="1.483287895s" totalImagesPullingTime="211.801513ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:04:29.481836956 +0000 UTC m=+849.793292175" watchObservedRunningTime="2026-07-06 20:04:29.483287895 +0000 UTC m=+849.794743114"
Jul 06 20:04:30.983438 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:30.983404    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:30.983951 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:30.983684    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:30.995590 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:30.995567    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:31.481647 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:31.481623    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98crx"
Jul 06 20:04:38.371958 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:38.371928    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:38.372450 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:38.371987    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:38.384555 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:38.384535    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:04:38.512492 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:04:38.512459    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d"
Jul 06 20:11:51.639685 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:51.639646    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"]
Jul 06 20:11:51.640296 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:51.639873    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5" podUID="e479ef2a-21df-490c-b995-bb3aa448a710" containerName="manager" containerID="cri-o://072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649" gracePeriod=10
Jul 06 20:11:54.286067 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.286042    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:11:54.322042 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.322011    2577 generic.go:356] "Generic (PLEG): container finished" podID="e479ef2a-21df-490c-b995-bb3aa448a710" containerID="072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649" exitCode=0
Jul 06 20:11:54.322329 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.322069    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"
Jul 06 20:11:54.322329 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.322093    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5" event={"ID":"e479ef2a-21df-490c-b995-bb3aa448a710","Type":"ContainerDied","Data":"072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649"}
Jul 06 20:11:54.322329 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.322156    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5" event={"ID":"e479ef2a-21df-490c-b995-bb3aa448a710","Type":"ContainerDied","Data":"8f506e7fe07921bf5e004b4efd224218d7cb8384d35de56f1d2f0f96af1b8b14"}
Jul 06 20:11:54.322329 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.322210    2577 scope.go:117] "RemoveContainer" containerID="072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649"
Jul 06 20:11:54.334375 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.334355    2577 scope.go:117] "RemoveContainer" containerID="072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649"
Jul 06 20:11:54.334687 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:11:54.334650    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649\": container with ID starting with 072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649 not found: ID does not exist" containerID="072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649"
Jul 06 20:11:54.334778 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.334699    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649"} err="failed to get container status \"072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649\": rpc error: code = NotFound desc = could not find container \"072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649\": container with ID starting with 072d3434540e26040374c326fb6db9973e87fab1fcbb1949a72e50896d945649 not found: ID does not exist"
Jul 06 20:11:54.376066 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.376037    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e479ef2a-21df-490c-b995-bb3aa448a710-extensions-socket-volume\") pod \"e479ef2a-21df-490c-b995-bb3aa448a710\" (UID: \"e479ef2a-21df-490c-b995-bb3aa448a710\") "
Jul 06 20:11:54.376268 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.376145    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtrgs\" (UniqueName: \"kubernetes.io/projected/e479ef2a-21df-490c-b995-bb3aa448a710-kube-api-access-gtrgs\") pod \"e479ef2a-21df-490c-b995-bb3aa448a710\" (UID: \"e479ef2a-21df-490c-b995-bb3aa448a710\") "
Jul 06 20:11:54.376453 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.376430    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e479ef2a-21df-490c-b995-bb3aa448a710-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e479ef2a-21df-490c-b995-bb3aa448a710" (UID: "e479ef2a-21df-490c-b995-bb3aa448a710"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:11:54.378217 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.378165    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e479ef2a-21df-490c-b995-bb3aa448a710-kube-api-access-gtrgs" (OuterVolumeSpecName: "kube-api-access-gtrgs") pod "e479ef2a-21df-490c-b995-bb3aa448a710" (UID: "e479ef2a-21df-490c-b995-bb3aa448a710"). InnerVolumeSpecName "kube-api-access-gtrgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:11:54.477525 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.477479    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtrgs\" (UniqueName: \"kubernetes.io/projected/e479ef2a-21df-490c-b995-bb3aa448a710-kube-api-access-gtrgs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:11:54.477525 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.477517    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e479ef2a-21df-490c-b995-bb3aa448a710-extensions-socket-volume\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:11:54.643831 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.643786    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"]
Jul 06 20:11:54.652504 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:54.647752    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6llx5"]
Jul 06 20:11:56.336542 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:11:56.336508    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e479ef2a-21df-490c-b995-bb3aa448a710" path="/var/lib/kubelet/pods/e479ef2a-21df-490c-b995-bb3aa448a710/volumes"
Jul 06 20:12:52.918788 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:52.918710    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"]
Jul 06 20:12:52.919239 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:52.919030    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e479ef2a-21df-490c-b995-bb3aa448a710" containerName="manager"
Jul 06 20:12:52.919239 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:52.919045    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e479ef2a-21df-490c-b995-bb3aa448a710" containerName="manager"
Jul 06 20:12:52.919239 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:52.919112    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e479ef2a-21df-490c-b995-bb3aa448a710" containerName="manager"
Jul 06 20:12:52.921955 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:52.921935    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"
Jul 06 20:12:52.924687 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:52.924667    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kvfcs\""
Jul 06 20:12:52.947773 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:52.947738    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"]
Jul 06 20:12:53.067695 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.067641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjnjw\" (UniqueName: \"kubernetes.io/projected/f2360391-eee4-4c7a-baa7-f7664d420110-kube-api-access-wjnjw\") pod \"kuadrant-operator-controller-manager-86878c769f-l557g\" (UID: \"f2360391-eee4-4c7a-baa7-f7664d420110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"
Jul 06 20:12:53.067695 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.067702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2360391-eee4-4c7a-baa7-f7664d420110-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l557g\" (UID: \"f2360391-eee4-4c7a-baa7-f7664d420110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"
Jul 06 20:12:53.168621 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.168579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjnjw\" (UniqueName: \"kubernetes.io/projected/f2360391-eee4-4c7a-baa7-f7664d420110-kube-api-access-wjnjw\") pod \"kuadrant-operator-controller-manager-86878c769f-l557g\" (UID: \"f2360391-eee4-4c7a-baa7-f7664d420110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"
Jul 06 20:12:53.168621 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.168627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2360391-eee4-4c7a-baa7-f7664d420110-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l557g\" (UID: \"f2360391-eee4-4c7a-baa7-f7664d420110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"
Jul 06 20:12:53.168998 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.168959    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2360391-eee4-4c7a-baa7-f7664d420110-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l557g\" (UID: \"f2360391-eee4-4c7a-baa7-f7664d420110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"
Jul 06 20:12:53.176859 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.176826    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjnjw\" (UniqueName: \"kubernetes.io/projected/f2360391-eee4-4c7a-baa7-f7664d420110-kube-api-access-wjnjw\") pod \"kuadrant-operator-controller-manager-86878c769f-l557g\" (UID: \"f2360391-eee4-4c7a-baa7-f7664d420110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"
Jul 06 20:12:53.231835 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.231795    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"
Jul 06 20:12:53.366833 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.366802    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"]
Jul 06 20:12:53.369426 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:12:53.369399    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2360391_eee4_4c7a_baa7_f7664d420110.slice/crio-aae5520203fa15b1abe2dfe49b8c4fbe436e43ce0780b3150204441fd822ec30 WatchSource:0}: Error finding container aae5520203fa15b1abe2dfe49b8c4fbe436e43ce0780b3150204441fd822ec30: Status 404 returned error can't find the container with id aae5520203fa15b1abe2dfe49b8c4fbe436e43ce0780b3150204441fd822ec30
Jul 06 20:12:53.569421 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.569388    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g" event={"ID":"f2360391-eee4-4c7a-baa7-f7664d420110","Type":"ContainerStarted","Data":"4f4aad544bc2455fa72280f1fe7de2120dc4542a9550ddf7214b13cdf105ff6c"}
Jul 06 20:12:53.569581 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.569439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g" event={"ID":"f2360391-eee4-4c7a-baa7-f7664d420110","Type":"ContainerStarted","Data":"aae5520203fa15b1abe2dfe49b8c4fbe436e43ce0780b3150204441fd822ec30"}
Jul 06 20:12:53.569581 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.569483    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"
Jul 06 20:12:53.593555 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:12:53.593493    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g" podStartSLOduration=1.5934779 podStartE2EDuration="1.5934779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:12:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:12:53.592160563 +0000 UTC m=+1353.903615786" watchObservedRunningTime="2026-07-06 20:12:53.5934779 +0000 UTC m=+1353.904933123"
Jul 06 20:13:04.576121 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:13:04.576090    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l557g"
Jul 06 20:19:10.120353 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.120315    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"]
Jul 06 20:19:10.123741 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.123712    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:10.126203 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.126160    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-782907f8-serving-cert\""
Jul 06 20:19:10.126333 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.126272    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mthr8\""
Jul 06 20:19:10.133838 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.133808    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"]
Jul 06 20:19:10.247480 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.247435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqbk2\" (UniqueName: \"kubernetes.io/projected/c0e9de33-3697-425f-9d84-6422083a6e78-kube-api-access-zqbk2\") pod \"maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") " pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:10.247682 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.247517    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0e9de33-3697-425f-9d84-6422083a6e78-maas-api-tls\") pod \"maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") " pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:10.247682 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.247540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0e9de33-3697-425f-9d84-6422083a6e78-openshift-service-ca\") pod \"maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") " pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:10.348500 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.348462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbk2\" (UniqueName: \"kubernetes.io/projected/c0e9de33-3697-425f-9d84-6422083a6e78-kube-api-access-zqbk2\") pod \"maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") " pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:10.348696 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.348529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0e9de33-3697-425f-9d84-6422083a6e78-maas-api-tls\") pod \"maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") " pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:10.348696 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.348553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0e9de33-3697-425f-9d84-6422083a6e78-openshift-service-ca\") pod \"maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") " pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:10.349227 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.349207    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0e9de33-3697-425f-9d84-6422083a6e78-openshift-service-ca\") pod \"maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") " pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:10.351040 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.351012    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0e9de33-3697-425f-9d84-6422083a6e78-maas-api-tls\") pod \"maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") " pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:10.357565 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.357541    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqbk2\" (UniqueName: \"kubernetes.io/projected/c0e9de33-3697-425f-9d84-6422083a6e78-kube-api-access-zqbk2\") pod \"maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") " pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:10.437612 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.437521    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:10.575435 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.575403    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"]
Jul 06 20:19:10.579599 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:19:10.579566    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0e9de33_3697_425f_9d84_6422083a6e78.slice/crio-0f69a2e6dd09ffec9f0071d3c71d9eb4272769795194449ae33607e0534910b1 WatchSource:0}: Error finding container 0f69a2e6dd09ffec9f0071d3c71d9eb4272769795194449ae33607e0534910b1: Status 404 returned error can't find the container with id 0f69a2e6dd09ffec9f0071d3c71d9eb4272769795194449ae33607e0534910b1
Jul 06 20:19:10.581099 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:10.581078    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:19:11.168579 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:11.168530    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6" event={"ID":"c0e9de33-3697-425f-9d84-6422083a6e78","Type":"ContainerStarted","Data":"0f69a2e6dd09ffec9f0071d3c71d9eb4272769795194449ae33607e0534910b1"}
Jul 06 20:19:13.180408 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:13.180376    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6" event={"ID":"c0e9de33-3697-425f-9d84-6422083a6e78","Type":"ContainerStarted","Data":"49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3"}
Jul 06 20:19:13.180792 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:13.180439    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:13.197335 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:13.197287    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6" podStartSLOduration=0.966206615 podStartE2EDuration="3.197270827s" totalImagesPullingTime="2.231064212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:19:13.195969354 +0000 UTC m=+1733.507424575" watchObservedRunningTime="2026-07-06 20:19:13.197270827 +0000 UTC m=+1733.508726048"
Jul 06 20:19:15.372208 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.372145    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"]
Jul 06 20:19:15.372681 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.372435    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6" podUID="c0e9de33-3697-425f-9d84-6422083a6e78" containerName="maas-api" containerID="cri-o://49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3" gracePeriod=30
Jul 06 20:19:15.618170 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.618136    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:15.696723 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.696629    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqbk2\" (UniqueName: \"kubernetes.io/projected/c0e9de33-3697-425f-9d84-6422083a6e78-kube-api-access-zqbk2\") pod \"c0e9de33-3697-425f-9d84-6422083a6e78\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") "
Jul 06 20:19:15.696723 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.696674    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0e9de33-3697-425f-9d84-6422083a6e78-openshift-service-ca\") pod \"c0e9de33-3697-425f-9d84-6422083a6e78\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") "
Jul 06 20:19:15.696723 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.696714    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0e9de33-3697-425f-9d84-6422083a6e78-maas-api-tls\") pod \"c0e9de33-3697-425f-9d84-6422083a6e78\" (UID: \"c0e9de33-3697-425f-9d84-6422083a6e78\") "
Jul 06 20:19:15.697081 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.697059    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0e9de33-3697-425f-9d84-6422083a6e78-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c0e9de33-3697-425f-9d84-6422083a6e78" (UID: "c0e9de33-3697-425f-9d84-6422083a6e78"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:19:15.698967 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.698941    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0e9de33-3697-425f-9d84-6422083a6e78-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c0e9de33-3697-425f-9d84-6422083a6e78" (UID: "c0e9de33-3697-425f-9d84-6422083a6e78"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:19:15.699022 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.698996    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0e9de33-3697-425f-9d84-6422083a6e78-kube-api-access-zqbk2" (OuterVolumeSpecName: "kube-api-access-zqbk2") pod "c0e9de33-3697-425f-9d84-6422083a6e78" (UID: "c0e9de33-3697-425f-9d84-6422083a6e78"). InnerVolumeSpecName "kube-api-access-zqbk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:19:15.797355 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.797314    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqbk2\" (UniqueName: \"kubernetes.io/projected/c0e9de33-3697-425f-9d84-6422083a6e78-kube-api-access-zqbk2\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:15.797355 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.797346    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0e9de33-3697-425f-9d84-6422083a6e78-openshift-service-ca\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:15.797355 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:15.797356    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0e9de33-3697-425f-9d84-6422083a6e78-maas-api-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:16.195747 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:16.195708    2577 generic.go:356] "Generic (PLEG): container finished" podID="c0e9de33-3697-425f-9d84-6422083a6e78" containerID="49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3" exitCode=0
Jul 06 20:19:16.195933 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:16.195755    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"
Jul 06 20:19:16.195933 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:16.195791    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6" event={"ID":"c0e9de33-3697-425f-9d84-6422083a6e78","Type":"ContainerDied","Data":"49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3"}
Jul 06 20:19:16.195933 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:16.195891    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6" event={"ID":"c0e9de33-3697-425f-9d84-6422083a6e78","Type":"ContainerDied","Data":"0f69a2e6dd09ffec9f0071d3c71d9eb4272769795194449ae33607e0534910b1"}
Jul 06 20:19:16.195933 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:16.195914    2577 scope.go:117] "RemoveContainer" containerID="49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3"
Jul 06 20:19:16.204509 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:16.204493    2577 scope.go:117] "RemoveContainer" containerID="49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3"
Jul 06 20:19:16.204760 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:19:16.204743    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3\": container with ID starting with 49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3 not found: ID does not exist" containerID="49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3"
Jul 06 20:19:16.204845 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:16.204768    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3"} err="failed to get container status \"49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3\": rpc error: code = NotFound desc = could not find container \"49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3\": container with ID starting with 49179d78dc1c2cc48885cf43cd8a246a6e15304f7e3f89a617fecc89f9399ad3 not found: ID does not exist"
Jul 06 20:19:16.217375 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:16.217344    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"]
Jul 06 20:19:16.219894 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:16.219869    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-782907f8-649d44b6b8-kbsb6"]
Jul 06 20:19:16.337506 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:16.337476    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0e9de33-3697-425f-9d84-6422083a6e78" path="/var/lib/kubelet/pods/c0e9de33-3697-425f-9d84-6422083a6e78/volumes"
Jul 06 20:19:21.053398 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.053363    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"]
Jul 06 20:19:21.053868 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.053851    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0e9de33-3697-425f-9d84-6422083a6e78" containerName="maas-api"
Jul 06 20:19:21.053912 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.053872    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0e9de33-3697-425f-9d84-6422083a6e78" containerName="maas-api"
Jul 06 20:19:21.053960 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.053948    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0e9de33-3697-425f-9d84-6422083a6e78" containerName="maas-api"
Jul 06 20:19:21.058450 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.058424    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.060963 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.060938    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 20:19:21.061136 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.061117    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-11e70696-openshift-default-dockercfg-xsq4j\""
Jul 06 20:19:21.070681 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.070611    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"]
Jul 06 20:19:21.147237 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.147201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-envoy\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.147401 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.147249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-socket\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.147401 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.147307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-data\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.147401 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.147328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-certs\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.147401 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.147355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-token\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.147532 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.147417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-credential-socket\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.147532 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.147453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e2a294-512f-4428-99e8-ab89551a46d6-istiod-ca-cert\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.147532 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.147476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jrpt\" (UniqueName: \"kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-kube-api-access-5jrpt\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.147532 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.147501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-podinfo\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248007 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.247972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-data\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248007 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-certs\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248296 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-token\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248296 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248051    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-credential-socket\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248296 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e2a294-512f-4428-99e8-ab89551a46d6-istiod-ca-cert\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248296 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jrpt\" (UniqueName: \"kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-kube-api-access-5jrpt\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248296 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-podinfo\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248296 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-envoy\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248296 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-socket\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248575 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248478    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-certs\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248650 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248622    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-socket\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248709 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248643    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-data\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.248765 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.248704    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-credential-socket\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.249033 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.249010    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e2a294-512f-4428-99e8-ab89551a46d6-istiod-ca-cert\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.250507 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.250487    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-envoy\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.250720 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.250699    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-podinfo\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.256104 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.256079    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-token\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.256220 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.256165    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jrpt\" (UniqueName: \"kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-kube-api-access-5jrpt\") pod \"e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") " pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.370933 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.370908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:21.508768 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:21.508728    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"]
Jul 06 20:19:21.511821 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:19:21.511793    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7e2a294_512f_4428_99e8_ab89551a46d6.slice/crio-78a8495134760559f807cc030760b2b6b35600293b1961d11264d878fbedf190 WatchSource:0}: Error finding container 78a8495134760559f807cc030760b2b6b35600293b1961d11264d878fbedf190: Status 404 returned error can't find the container with id 78a8495134760559f807cc030760b2b6b35600293b1961d11264d878fbedf190
Jul 06 20:19:22.227421 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:22.227384    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl" event={"ID":"c7e2a294-512f-4428-99e8-ab89551a46d6","Type":"ContainerStarted","Data":"78a8495134760559f807cc030760b2b6b35600293b1961d11264d878fbedf190"}
Jul 06 20:19:24.110304 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:24.110259    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:19:24.110602 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:24.110360    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:19:24.110602 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:24.110401    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:19:24.239031 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:24.238995    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl" event={"ID":"c7e2a294-512f-4428-99e8-ab89551a46d6","Type":"ContainerStarted","Data":"266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6"}
Jul 06 20:19:24.258934 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:24.258870    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl" podStartSLOduration=0.662990675 podStartE2EDuration="3.258848126s" totalImagesPullingTime="2.595857451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:19:24.25769499 +0000 UTC m=+1744.569150215" watchObservedRunningTime="2026-07-06 20:19:24.258848126 +0000 UTC m=+1744.570303351"
Jul 06 20:19:24.371676 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:24.371649    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:25.376237 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:25.376198    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:26.248696 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:26.248666    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:26.249818 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:26.249798    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:33.278897 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:33.278867    2577 generic.go:356] "Generic (PLEG): container finished" podID="c7e2a294-512f-4428-99e8-ab89551a46d6" containerID="266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6" exitCode=0
Jul 06 20:19:33.279341 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:33.278926    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl" event={"ID":"c7e2a294-512f-4428-99e8-ab89551a46d6","Type":"ContainerDied","Data":"266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6"}
Jul 06 20:19:33.279436 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:33.279419    2577 scope.go:117] "RemoveContainer" containerID="266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6"
Jul 06 20:19:33.281864 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:33.281832    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:19:33.281957 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:33.281907    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:19:33.281957 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:33.281939    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:19:33.326434 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:33.326401    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"]
Jul 06 20:19:34.285484 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:34.285449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl" event={"ID":"c7e2a294-512f-4428-99e8-ab89551a46d6","Type":"ContainerStarted","Data":"8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043"}
Jul 06 20:19:34.285948 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:34.285564    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl" podUID="c7e2a294-512f-4428-99e8-ab89551a46d6" containerName="istio-proxy" containerID="cri-o://8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043" gracePeriod=30
Jul 06 20:19:39.545668 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.545612    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:39.621376 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621342    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-envoy\") pod \"c7e2a294-512f-4428-99e8-ab89551a46d6\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") "
Jul 06 20:19:39.621664 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621383    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-credential-socket\") pod \"c7e2a294-512f-4428-99e8-ab89551a46d6\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") "
Jul 06 20:19:39.621664 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621412    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-certs\") pod \"c7e2a294-512f-4428-99e8-ab89551a46d6\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") "
Jul 06 20:19:39.621664 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621435    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-socket\") pod \"c7e2a294-512f-4428-99e8-ab89551a46d6\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") "
Jul 06 20:19:39.621664 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621520    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-data\") pod \"c7e2a294-512f-4428-99e8-ab89551a46d6\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") "
Jul 06 20:19:39.621664 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621550    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-podinfo\") pod \"c7e2a294-512f-4428-99e8-ab89551a46d6\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") "
Jul 06 20:19:39.621664 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621581    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jrpt\" (UniqueName: \"kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-kube-api-access-5jrpt\") pod \"c7e2a294-512f-4428-99e8-ab89551a46d6\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") "
Jul 06 20:19:39.621664 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621608    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-token\") pod \"c7e2a294-512f-4428-99e8-ab89551a46d6\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") "
Jul 06 20:19:39.621664 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621634    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e2a294-512f-4428-99e8-ab89551a46d6-istiod-ca-cert\") pod \"c7e2a294-512f-4428-99e8-ab89551a46d6\" (UID: \"c7e2a294-512f-4428-99e8-ab89551a46d6\") "
Jul 06 20:19:39.622062 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621685    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c7e2a294-512f-4428-99e8-ab89551a46d6" (UID: "c7e2a294-512f-4428-99e8-ab89551a46d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:19:39.622062 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621730    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c7e2a294-512f-4428-99e8-ab89551a46d6" (UID: "c7e2a294-512f-4428-99e8-ab89551a46d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:19:39.622062 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621823    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c7e2a294-512f-4428-99e8-ab89551a46d6" (UID: "c7e2a294-512f-4428-99e8-ab89551a46d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:19:39.622062 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621949    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-credential-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:39.622062 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621967    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-certs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:39.622062 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.621982    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-workload-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:39.622062 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.622041    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7e2a294-512f-4428-99e8-ab89551a46d6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c7e2a294-512f-4428-99e8-ab89551a46d6" (UID: "c7e2a294-512f-4428-99e8-ab89551a46d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:19:39.622451 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.622424    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-data" (OuterVolumeSpecName: "istio-data") pod "c7e2a294-512f-4428-99e8-ab89551a46d6" (UID: "c7e2a294-512f-4428-99e8-ab89551a46d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:19:39.624292 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.624268    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-kube-api-access-5jrpt" (OuterVolumeSpecName: "kube-api-access-5jrpt") pod "c7e2a294-512f-4428-99e8-ab89551a46d6" (UID: "c7e2a294-512f-4428-99e8-ab89551a46d6"). InnerVolumeSpecName "kube-api-access-5jrpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:19:39.624292 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.624286    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "c7e2a294-512f-4428-99e8-ab89551a46d6" (UID: "c7e2a294-512f-4428-99e8-ab89551a46d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:19:39.624453 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.624279    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c7e2a294-512f-4428-99e8-ab89551a46d6" (UID: "c7e2a294-512f-4428-99e8-ab89551a46d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 20:19:39.624453 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.624270    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c7e2a294-512f-4428-99e8-ab89551a46d6" (UID: "c7e2a294-512f-4428-99e8-ab89551a46d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:19:39.722483 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.722450    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-data\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:39.722483 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.722480    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-podinfo\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:39.722483 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.722489    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jrpt\" (UniqueName: \"kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-kube-api-access-5jrpt\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:39.722721 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.722498    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-token\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:39.722721 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.722506    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e2a294-512f-4428-99e8-ab89551a46d6-istiod-ca-cert\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:39.722721 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:39.722514    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e2a294-512f-4428-99e8-ab89551a46d6-istio-envoy\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:40.313222 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.313164    2577 generic.go:356] "Generic (PLEG): container finished" podID="c7e2a294-512f-4428-99e8-ab89551a46d6" containerID="8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043" exitCode=0
Jul 06 20:19:40.313222 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.313201    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl" event={"ID":"c7e2a294-512f-4428-99e8-ab89551a46d6","Type":"ContainerDied","Data":"8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043"}
Jul 06 20:19:40.313222 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.313226    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"
Jul 06 20:19:40.313504 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.313265    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl" event={"ID":"c7e2a294-512f-4428-99e8-ab89551a46d6","Type":"ContainerDied","Data":"78a8495134760559f807cc030760b2b6b35600293b1961d11264d878fbedf190"}
Jul 06 20:19:40.313504 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.313293    2577 scope.go:117] "RemoveContainer" containerID="8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043"
Jul 06 20:19:40.322256 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.322238    2577 scope.go:117] "RemoveContainer" containerID="266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6"
Jul 06 20:19:40.329511 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.329495    2577 scope.go:117] "RemoveContainer" containerID="8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043"
Jul 06 20:19:40.329790 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:19:40.329770    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043\": container with ID starting with 8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043 not found: ID does not exist" containerID="8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043"
Jul 06 20:19:40.329844 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.329800    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043"} err="failed to get container status \"8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043\": rpc error: code = NotFound desc = could not find container \"8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043\": container with ID starting with 8754addcce4496a04eb6cf71e58867e2f9531e1ed0cf31e623667a6ba4567043 not found: ID does not exist"
Jul 06 20:19:40.329844 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.329819    2577 scope.go:117] "RemoveContainer" containerID="266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6"
Jul 06 20:19:40.330062 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:19:40.330045    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6\": container with ID starting with 266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6 not found: ID does not exist" containerID="266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6"
Jul 06 20:19:40.330114 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.330066    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6"} err="failed to get container status \"266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6\": rpc error: code = NotFound desc = could not find container \"266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6\": container with ID starting with 266525b2ad53117a42be50b9939a7f635a435fc04f06e69b69edb43c2eb02ab6 not found: ID does not exist"
Jul 06 20:19:40.338868 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.338849    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"]
Jul 06 20:19:40.338964 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.338877    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-11e70696-openshift-default-5f966fcf45-zpspl"]
Jul 06 20:19:40.814281 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.814238    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"]
Jul 06 20:19:40.814775 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.814756    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7e2a294-512f-4428-99e8-ab89551a46d6" containerName="istio-proxy"
Jul 06 20:19:40.814825 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.814782    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7e2a294-512f-4428-99e8-ab89551a46d6" containerName="istio-proxy"
Jul 06 20:19:40.814894 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.814882    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7e2a294-512f-4428-99e8-ab89551a46d6" containerName="istio-proxy"
Jul 06 20:19:40.819164 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.819143    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:40.821796 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.821774    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mthr8\""
Jul 06 20:19:40.821931 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.821912    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-373ca5d7-serving-cert\""
Jul 06 20:19:40.830704 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.830666    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"]
Jul 06 20:19:40.831057 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.831034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13726b21-fe6c-4545-9eed-6f739ce449a9-maas-api-tls\") pod \"maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") " pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:40.831162 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.831118    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13726b21-fe6c-4545-9eed-6f739ce449a9-openshift-service-ca\") pod \"maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") " pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:40.831243 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.831198    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-677hx\" (UniqueName: \"kubernetes.io/projected/13726b21-fe6c-4545-9eed-6f739ce449a9-kube-api-access-677hx\") pod \"maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") " pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:40.931878 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.931849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-677hx\" (UniqueName: \"kubernetes.io/projected/13726b21-fe6c-4545-9eed-6f739ce449a9-kube-api-access-677hx\") pod \"maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") " pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:40.932039 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.931894    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13726b21-fe6c-4545-9eed-6f739ce449a9-maas-api-tls\") pod \"maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") " pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:40.932039 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.931966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13726b21-fe6c-4545-9eed-6f739ce449a9-openshift-service-ca\") pod \"maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") " pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:40.932654 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.932625    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13726b21-fe6c-4545-9eed-6f739ce449a9-openshift-service-ca\") pod \"maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") " pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:40.934326 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.934305    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13726b21-fe6c-4545-9eed-6f739ce449a9-maas-api-tls\") pod \"maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") " pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:40.939819 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:40.939789    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-677hx\" (UniqueName: \"kubernetes.io/projected/13726b21-fe6c-4545-9eed-6f739ce449a9-kube-api-access-677hx\") pod \"maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") " pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:41.129800 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:41.129770    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:41.266756 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:41.266722    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"]
Jul 06 20:19:41.269749 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:19:41.269718    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13726b21_fe6c_4545_9eed_6f739ce449a9.slice/crio-9a9512baf3b40334b2135ea3b91a6c4b2407f9a221c0c84976fdc17306eb4f9a WatchSource:0}: Error finding container 9a9512baf3b40334b2135ea3b91a6c4b2407f9a221c0c84976fdc17306eb4f9a: Status 404 returned error can't find the container with id 9a9512baf3b40334b2135ea3b91a6c4b2407f9a221c0c84976fdc17306eb4f9a
Jul 06 20:19:41.318720 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:41.318676    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm" event={"ID":"13726b21-fe6c-4545-9eed-6f739ce449a9","Type":"ContainerStarted","Data":"9a9512baf3b40334b2135ea3b91a6c4b2407f9a221c0c84976fdc17306eb4f9a"}
Jul 06 20:19:42.325818 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:42.325780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm" event={"ID":"13726b21-fe6c-4545-9eed-6f739ce449a9","Type":"ContainerStarted","Data":"3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7"}
Jul 06 20:19:42.326232 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:42.325868    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:42.337154 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:42.337126    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7e2a294-512f-4428-99e8-ab89551a46d6" path="/var/lib/kubelet/pods/c7e2a294-512f-4428-99e8-ab89551a46d6/volumes"
Jul 06 20:19:42.344144 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:42.344090    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm" podStartSLOduration=1.778828795 podStartE2EDuration="2.34406981s" totalImagesPullingTime="565.241015ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:19:42.342047095 +0000 UTC m=+1762.653502315" watchObservedRunningTime="2026-07-06 20:19:42.34406981 +0000 UTC m=+1762.655525034"
Jul 06 20:19:45.535441 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.535404    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"]
Jul 06 20:19:45.535813 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.535604    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm" podUID="13726b21-fe6c-4545-9eed-6f739ce449a9" containerName="maas-api" containerID="cri-o://3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7" gracePeriod=30
Jul 06 20:19:45.776618 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.776595    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:45.877367 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.877341    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13726b21-fe6c-4545-9eed-6f739ce449a9-openshift-service-ca\") pod \"13726b21-fe6c-4545-9eed-6f739ce449a9\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") "
Jul 06 20:19:45.877516 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.877420    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13726b21-fe6c-4545-9eed-6f739ce449a9-maas-api-tls\") pod \"13726b21-fe6c-4545-9eed-6f739ce449a9\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") "
Jul 06 20:19:45.877516 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.877444    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-677hx\" (UniqueName: \"kubernetes.io/projected/13726b21-fe6c-4545-9eed-6f739ce449a9-kube-api-access-677hx\") pod \"13726b21-fe6c-4545-9eed-6f739ce449a9\" (UID: \"13726b21-fe6c-4545-9eed-6f739ce449a9\") "
Jul 06 20:19:45.877675 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.877651    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13726b21-fe6c-4545-9eed-6f739ce449a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13726b21-fe6c-4545-9eed-6f739ce449a9" (UID: "13726b21-fe6c-4545-9eed-6f739ce449a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:19:45.879528 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.879490    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13726b21-fe6c-4545-9eed-6f739ce449a9-kube-api-access-677hx" (OuterVolumeSpecName: "kube-api-access-677hx") pod "13726b21-fe6c-4545-9eed-6f739ce449a9" (UID: "13726b21-fe6c-4545-9eed-6f739ce449a9"). InnerVolumeSpecName "kube-api-access-677hx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:19:45.879635 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.879600    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13726b21-fe6c-4545-9eed-6f739ce449a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13726b21-fe6c-4545-9eed-6f739ce449a9" (UID: "13726b21-fe6c-4545-9eed-6f739ce449a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:19:45.978803 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.978768    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13726b21-fe6c-4545-9eed-6f739ce449a9-maas-api-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:45.978803 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.978797    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-677hx\" (UniqueName: \"kubernetes.io/projected/13726b21-fe6c-4545-9eed-6f739ce449a9-kube-api-access-677hx\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:45.978803 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:45.978808    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13726b21-fe6c-4545-9eed-6f739ce449a9-openshift-service-ca\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:19:46.343100 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:46.343020    2577 generic.go:356] "Generic (PLEG): container finished" podID="13726b21-fe6c-4545-9eed-6f739ce449a9" containerID="3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7" exitCode=0
Jul 06 20:19:46.343100 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:46.343057    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm" event={"ID":"13726b21-fe6c-4545-9eed-6f739ce449a9","Type":"ContainerDied","Data":"3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7"}
Jul 06 20:19:46.343100 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:46.343078    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"
Jul 06 20:19:46.343100 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:46.343096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm" event={"ID":"13726b21-fe6c-4545-9eed-6f739ce449a9","Type":"ContainerDied","Data":"9a9512baf3b40334b2135ea3b91a6c4b2407f9a221c0c84976fdc17306eb4f9a"}
Jul 06 20:19:46.343401 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:46.343119    2577 scope.go:117] "RemoveContainer" containerID="3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7"
Jul 06 20:19:46.352158 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:46.352138    2577 scope.go:117] "RemoveContainer" containerID="3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7"
Jul 06 20:19:46.352465 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:19:46.352441    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7\": container with ID starting with 3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7 not found: ID does not exist" containerID="3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7"
Jul 06 20:19:46.352560 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:46.352471    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7"} err="failed to get container status \"3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7\": rpc error: code = NotFound desc = could not find container \"3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7\": container with ID starting with 3f5bb037e59b2d291e10b07bb3b7ac651b6b232378aa140688bd795036c500c7 not found: ID does not exist"
Jul 06 20:19:46.360929 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:46.360904    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"]
Jul 06 20:19:46.364312 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:46.364292    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-373ca5d7-86cd996b99-h25xm"]
Jul 06 20:19:48.337407 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:19:48.337377    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13726b21-fe6c-4545-9eed-6f739ce449a9" path="/var/lib/kubelet/pods/13726b21-fe6c-4545-9eed-6f739ce449a9/volumes"
Jul 06 20:21:26.118206 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.118156    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"]
Jul 06 20:21:26.118907 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.118525    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7e2a294-512f-4428-99e8-ab89551a46d6" containerName="istio-proxy"
Jul 06 20:21:26.118907 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.118537    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7e2a294-512f-4428-99e8-ab89551a46d6" containerName="istio-proxy"
Jul 06 20:21:26.118907 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.118546    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13726b21-fe6c-4545-9eed-6f739ce449a9" containerName="maas-api"
Jul 06 20:21:26.118907 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.118551    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="13726b21-fe6c-4545-9eed-6f739ce449a9" containerName="maas-api"
Jul 06 20:21:26.118907 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.118600    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7e2a294-512f-4428-99e8-ab89551a46d6" containerName="istio-proxy"
Jul 06 20:21:26.118907 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.118613    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="13726b21-fe6c-4545-9eed-6f739ce449a9" containerName="maas-api"
Jul 06 20:21:26.120625 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.120610    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.124514 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.124490    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7fb0e13c-openshift-default-dockercfg-m26x2\""
Jul 06 20:21:26.124826 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.124811    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 20:21:26.131844 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.131823    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-envoy\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.131928 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.131866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-podinfo\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.131928 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.131898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-certs\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.131928 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.131919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istiod-ca-cert\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.132028 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.132018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-token\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.132063 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.132041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79p75\" (UniqueName: \"kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-kube-api-access-79p75\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.132101 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.132061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-credential-socket\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.132143 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.132123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-socket\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.132211 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.132194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-data\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.188823 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.188795    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"]
Jul 06 20:21:26.232601 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.232568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-certs\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.232601 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.232601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istiod-ca-cert\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.232834 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.232640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-token\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.232834 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.232658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79p75\" (UniqueName: \"kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-kube-api-access-79p75\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.232834 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.232683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-credential-socket\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.232834 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.232723    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-socket\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.232834 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.232761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-data\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.232834 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.232785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-envoy\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.232834 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.232825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-podinfo\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.233202 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.233041    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-certs\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.233202 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.233116    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-credential-socket\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.233202 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.233168    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-socket\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.233314 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.233243    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-data\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.233549 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.233530    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istiod-ca-cert\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.235042 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.235015    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-envoy\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.235251 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.235231    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-podinfo\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.240614 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.240594    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-token\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.241005 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.240986    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79p75\" (UniqueName: \"kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-kube-api-access-79p75\") pod \"e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") " pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.430393 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.430363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:26.572836 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.572701    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"]
Jul 06 20:21:26.575387 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:21:26.575361    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f5d57b4_6f22_4387_aad1_cfa60c21c745.slice/crio-c0277cc70bc99f27ff9fb0ba413b1b8a3ed1611844f6d35d99fda1be9ce7102c WatchSource:0}: Error finding container c0277cc70bc99f27ff9fb0ba413b1b8a3ed1611844f6d35d99fda1be9ce7102c: Status 404 returned error can't find the container with id c0277cc70bc99f27ff9fb0ba413b1b8a3ed1611844f6d35d99fda1be9ce7102c
Jul 06 20:21:26.577895 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.577742    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:21:26.577895 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.577826    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:21:26.577895 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.577870    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:21:26.766868 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.766779    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" event={"ID":"0f5d57b4-6f22-4387-aad1-cfa60c21c745","Type":"ContainerStarted","Data":"e0e6eae8911ee743372abd464189fabd43d6404f0fd4d1472378d0fd8fdf86db"}
Jul 06 20:21:26.766868 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.766844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" event={"ID":"0f5d57b4-6f22-4387-aad1-cfa60c21c745","Type":"ContainerStarted","Data":"c0277cc70bc99f27ff9fb0ba413b1b8a3ed1611844f6d35d99fda1be9ce7102c"}
Jul 06 20:21:26.791873 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:26.791821    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" podStartSLOduration=0.791805141 podStartE2EDuration="791.805141ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:26.789890308 +0000 UTC m=+1867.101345530" watchObservedRunningTime="2026-07-06 20:21:26.791805141 +0000 UTC m=+1867.103260361"
Jul 06 20:21:27.430693 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:27.430660    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:27.436052 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:27.436027    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:27.771508 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:27.771427    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:27.772283 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:27.772269    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:28.820007 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.819961    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"]
Jul 06 20:21:28.822388 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.822365    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:28.824933 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.824909    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mthr8\""
Jul 06 20:21:28.825036 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.824916    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-7fb0e13c-serving-cert\""
Jul 06 20:21:28.839086 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.839063    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"]
Jul 06 20:21:28.851378 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.851353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg488\" (UniqueName: \"kubernetes.io/projected/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-kube-api-access-hg488\") pod \"maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") " pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:28.851496 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.851387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-maas-api-tls\") pod \"maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") " pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:28.851560 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.851539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-openshift-service-ca\") pod \"maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") " pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:28.952135 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.952107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-openshift-service-ca\") pod \"maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") " pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:28.952352 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.952165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg488\" (UniqueName: \"kubernetes.io/projected/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-kube-api-access-hg488\") pod \"maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") " pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:28.952352 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.952211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-maas-api-tls\") pod \"maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") " pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:28.952352 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:21:28.952308    2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-7fb0e13c-serving-cert: secret "maas-api-e2e-mt-7fb0e13c-serving-cert" not found
Jul 06 20:21:28.952502 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:21:28.952374    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-maas-api-tls podName:3af568c4-4f75-4ef5-bb36-5aa3e6c65190 nodeName:}" failed. No retries permitted until 2026-07-06 20:21:29.452352337 +0000 UTC m=+1869.763807554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-maas-api-tls") pod "maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8" (UID: "3af568c4-4f75-4ef5-bb36-5aa3e6c65190") : secret "maas-api-e2e-mt-7fb0e13c-serving-cert" not found
Jul 06 20:21:28.952826 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.952806    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-openshift-service-ca\") pod \"maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") " pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:28.960660 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:28.960638    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg488\" (UniqueName: \"kubernetes.io/projected/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-kube-api-access-hg488\") pod \"maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") " pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:29.457602 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:29.457567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-maas-api-tls\") pod \"maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") " pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:29.459988 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:29.459965    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-maas-api-tls\") pod \"maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") " pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:29.732445 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:29.732354    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:29.863842 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:29.863814    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"]
Jul 06 20:21:29.866142 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:21:29.866117    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3af568c4_4f75_4ef5_bb36_5aa3e6c65190.slice/crio-4f3775751200a32a572aee2bea148e8c632cd60079c811de78288b9584cdb3eb WatchSource:0}: Error finding container 4f3775751200a32a572aee2bea148e8c632cd60079c811de78288b9584cdb3eb: Status 404 returned error can't find the container with id 4f3775751200a32a572aee2bea148e8c632cd60079c811de78288b9584cdb3eb
Jul 06 20:21:30.785955 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:30.785869    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8" event={"ID":"3af568c4-4f75-4ef5-bb36-5aa3e6c65190","Type":"ContainerStarted","Data":"81613185cff81d33774a264458ed9d45d5141e643f95f830fcf940c905901076"}
Jul 06 20:21:30.785955 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:30.785920    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8" event={"ID":"3af568c4-4f75-4ef5-bb36-5aa3e6c65190","Type":"ContainerStarted","Data":"4f3775751200a32a572aee2bea148e8c632cd60079c811de78288b9584cdb3eb"}
Jul 06 20:21:30.785955 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:30.785953    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:30.804369 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:30.804321    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8" podStartSLOduration=2.27643009 podStartE2EDuration="2.804306314s" totalImagesPullingTime="527.876224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:30.802270535 +0000 UTC m=+1871.113725755" watchObservedRunningTime="2026-07-06 20:21:30.804306314 +0000 UTC m=+1871.115761535"
Jul 06 20:21:36.794685 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:36.794659    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:40.658334 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:40.658299    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"]
Jul 06 20:21:40.658793 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:40.658572    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8" podUID="3af568c4-4f75-4ef5-bb36-5aa3e6c65190" containerName="maas-api" containerID="cri-o://81613185cff81d33774a264458ed9d45d5141e643f95f830fcf940c905901076" gracePeriod=30
Jul 06 20:21:40.846656 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:40.846607    2577 generic.go:356] "Generic (PLEG): container finished" podID="3af568c4-4f75-4ef5-bb36-5aa3e6c65190" containerID="81613185cff81d33774a264458ed9d45d5141e643f95f830fcf940c905901076" exitCode=0
Jul 06 20:21:40.846819 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:40.846718    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8" event={"ID":"3af568c4-4f75-4ef5-bb36-5aa3e6c65190","Type":"ContainerDied","Data":"81613185cff81d33774a264458ed9d45d5141e643f95f830fcf940c905901076"}
Jul 06 20:21:40.912049 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:40.911985    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:40.976542 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:40.976513    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-maas-api-tls\") pod \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") "
Jul 06 20:21:40.976740 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:40.976587    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg488\" (UniqueName: \"kubernetes.io/projected/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-kube-api-access-hg488\") pod \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") "
Jul 06 20:21:40.976740 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:40.976624    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-openshift-service-ca\") pod \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\" (UID: \"3af568c4-4f75-4ef5-bb36-5aa3e6c65190\") "
Jul 06 20:21:40.977053 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:40.977027    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3af568c4-4f75-4ef5-bb36-5aa3e6c65190" (UID: "3af568c4-4f75-4ef5-bb36-5aa3e6c65190"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:21:40.978630 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:40.978593    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3af568c4-4f75-4ef5-bb36-5aa3e6c65190" (UID: "3af568c4-4f75-4ef5-bb36-5aa3e6c65190"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:21:40.978630 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:40.978602    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-kube-api-access-hg488" (OuterVolumeSpecName: "kube-api-access-hg488") pod "3af568c4-4f75-4ef5-bb36-5aa3e6c65190" (UID: "3af568c4-4f75-4ef5-bb36-5aa3e6c65190"). InnerVolumeSpecName "kube-api-access-hg488". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:21:41.077551 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:41.077526    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-maas-api-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:41.077551 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:41.077548    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg488\" (UniqueName: \"kubernetes.io/projected/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-kube-api-access-hg488\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:41.077551 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:41.077558    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af568c4-4f75-4ef5-bb36-5aa3e6c65190-openshift-service-ca\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:41.853870 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:41.852772    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8" event={"ID":"3af568c4-4f75-4ef5-bb36-5aa3e6c65190","Type":"ContainerDied","Data":"4f3775751200a32a572aee2bea148e8c632cd60079c811de78288b9584cdb3eb"}
Jul 06 20:21:41.853870 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:41.852805    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"
Jul 06 20:21:41.853870 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:41.852833    2577 scope.go:117] "RemoveContainer" containerID="81613185cff81d33774a264458ed9d45d5141e643f95f830fcf940c905901076"
Jul 06 20:21:41.878243 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:41.878213    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"]
Jul 06 20:21:41.882512 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:41.882488    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-7fb0e13c-6594c5d957-mccb8"]
Jul 06 20:21:42.337760 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:42.337730    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3af568c4-4f75-4ef5-bb36-5aa3e6c65190" path="/var/lib/kubelet/pods/3af568c4-4f75-4ef5-bb36-5aa3e6c65190/volumes"
Jul 06 20:21:42.772611 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:42.772573    2577 patch_prober.go:28] interesting pod/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 06 20:21:42.772795 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:42.772634    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 06 20:21:42.858677 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:42.858647    2577 generic.go:356] "Generic (PLEG): container finished" podID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerID="e0e6eae8911ee743372abd464189fabd43d6404f0fd4d1472378d0fd8fdf86db" exitCode=0
Jul 06 20:21:42.859080 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:42.858719    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" event={"ID":"0f5d57b4-6f22-4387-aad1-cfa60c21c745","Type":"ContainerDied","Data":"e0e6eae8911ee743372abd464189fabd43d6404f0fd4d1472378d0fd8fdf86db"}
Jul 06 20:21:42.859187 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:42.859160    2577 scope.go:117] "RemoveContainer" containerID="e0e6eae8911ee743372abd464189fabd43d6404f0fd4d1472378d0fd8fdf86db"
Jul 06 20:21:42.860231 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:42.860202    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:21:42.860308 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:42.860275    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:21:42.860345 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:42.860306    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:21:43.865434 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:43.865402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" event={"ID":"0f5d57b4-6f22-4387-aad1-cfa60c21c745","Type":"ContainerStarted","Data":"d91c0dbbfa787b019ebde470a1416cad55f387d43a8b4a149552457da7f4dded"}
Jul 06 20:21:44.431783 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:44.431013    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:44.431783 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:44.431469    2577 patch_prober.go:28] interesting pod/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 06 20:21:44.431783 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:44.431523    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 06 20:21:45.431425 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:45.431374    2577 patch_prober.go:28] interesting pod/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 06 20:21:45.431930 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:45.431446    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 06 20:21:46.431853 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:46.431814    2577 patch_prober.go:28] interesting pod/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 06 20:21:46.432477 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:46.432443    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 06 20:21:46.889077 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:46.889040    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"]
Jul 06 20:21:46.889308 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:46.889286    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerName="istio-proxy" containerID="cri-o://d91c0dbbfa787b019ebde470a1416cad55f387d43a8b4a149552457da7f4dded" gracePeriod=30
Jul 06 20:21:52.912560 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:52.912524    2577 generic.go:356] "Generic (PLEG): container finished" podID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerID="d91c0dbbfa787b019ebde470a1416cad55f387d43a8b4a149552457da7f4dded" exitCode=0
Jul 06 20:21:52.913005 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:52.912600    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" event={"ID":"0f5d57b4-6f22-4387-aad1-cfa60c21c745","Type":"ContainerDied","Data":"d91c0dbbfa787b019ebde470a1416cad55f387d43a8b4a149552457da7f4dded"}
Jul 06 20:21:52.913005 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:52.912689    2577 scope.go:117] "RemoveContainer" containerID="e0e6eae8911ee743372abd464189fabd43d6404f0fd4d1472378d0fd8fdf86db"
Jul 06 20:21:52.931825 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:21:52.931802    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0e6eae8911ee743372abd464189fabd43d6404f0fd4d1472378d0fd8fdf86db\": container with ID starting with e0e6eae8911ee743372abd464189fabd43d6404f0fd4d1472378d0fd8fdf86db not found: ID does not exist" containerID="e0e6eae8911ee743372abd464189fabd43d6404f0fd4d1472378d0fd8fdf86db"
Jul 06 20:21:52.933545 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:52.933528    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:53.096879 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.096787    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-socket\") pod \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") "
Jul 06 20:21:53.096879 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.096831    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-data\") pod \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") "
Jul 06 20:21:53.096879 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.096850    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-token\") pod \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") "
Jul 06 20:21:53.097168 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.096893    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-envoy\") pod \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") "
Jul 06 20:21:53.097168 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.096909    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-certs\") pod \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") "
Jul 06 20:21:53.097168 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.096925    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istiod-ca-cert\") pod \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") "
Jul 06 20:21:53.097168 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.096960    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-podinfo\") pod \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") "
Jul 06 20:21:53.097168 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.097139    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0f5d57b4-6f22-4387-aad1-cfa60c21c745" (UID: "0f5d57b4-6f22-4387-aad1-cfa60c21c745"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:21:53.097168 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.097163    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-credential-socket\") pod \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") "
Jul 06 20:21:53.097489 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.097267    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79p75\" (UniqueName: \"kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-kube-api-access-79p75\") pod \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\" (UID: \"0f5d57b4-6f22-4387-aad1-cfa60c21c745\") "
Jul 06 20:21:53.097489 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.097301    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0f5d57b4-6f22-4387-aad1-cfa60c21c745" (UID: "0f5d57b4-6f22-4387-aad1-cfa60c21c745"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:21:53.097489 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.097397    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0f5d57b4-6f22-4387-aad1-cfa60c21c745" (UID: "0f5d57b4-6f22-4387-aad1-cfa60c21c745"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:21:53.097657 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.097486    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0f5d57b4-6f22-4387-aad1-cfa60c21c745" (UID: "0f5d57b4-6f22-4387-aad1-cfa60c21c745"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:21:53.097708 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.097679    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:53.097708 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.097694    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-workload-certs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:53.097708 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.097703    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istiod-ca-cert\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:53.097852 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.097711    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-credential-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:53.097904 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.097875    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-data" (OuterVolumeSpecName: "istio-data") pod "0f5d57b4-6f22-4387-aad1-cfa60c21c745" (UID: "0f5d57b4-6f22-4387-aad1-cfa60c21c745"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:21:53.099347 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.099314    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-token" (OuterVolumeSpecName: "istio-token") pod "0f5d57b4-6f22-4387-aad1-cfa60c21c745" (UID: "0f5d57b4-6f22-4387-aad1-cfa60c21c745"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:21:53.099445 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.099402    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-kube-api-access-79p75" (OuterVolumeSpecName: "kube-api-access-79p75") pod "0f5d57b4-6f22-4387-aad1-cfa60c21c745" (UID: "0f5d57b4-6f22-4387-aad1-cfa60c21c745"). InnerVolumeSpecName "kube-api-access-79p75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:21:53.099598 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.099579    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0f5d57b4-6f22-4387-aad1-cfa60c21c745" (UID: "0f5d57b4-6f22-4387-aad1-cfa60c21c745"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:21:53.099660 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.099578    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0f5d57b4-6f22-4387-aad1-cfa60c21c745" (UID: "0f5d57b4-6f22-4387-aad1-cfa60c21c745"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 20:21:53.198917 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.198885    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79p75\" (UniqueName: \"kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-kube-api-access-79p75\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:53.198917 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.198913    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-data\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:53.198917 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.198924    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-token\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:53.199145 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.198932    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-envoy\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:53.199145 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.198940    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f5d57b4-6f22-4387-aad1-cfa60c21c745-istio-podinfo\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:21:53.917824 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.917779    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5" event={"ID":"0f5d57b4-6f22-4387-aad1-cfa60c21c745","Type":"ContainerDied","Data":"c0277cc70bc99f27ff9fb0ba413b1b8a3ed1611844f6d35d99fda1be9ce7102c"}
Jul 06 20:21:53.918220 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.917839    2577 scope.go:117] "RemoveContainer" containerID="d91c0dbbfa787b019ebde470a1416cad55f387d43a8b4a149552457da7f4dded"
Jul 06 20:21:53.918220 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.917802    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"
Jul 06 20:21:53.955654 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.955615    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"]
Jul 06 20:21:53.969834 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:53.969801    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7fb0e13c-openshift-default-5dcbffc8cf-pc8r5"]
Jul 06 20:21:54.338439 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:21:54.338364    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" path="/var/lib/kubelet/pods/0f5d57b4-6f22-4387-aad1-cfa60c21c745/volumes"
Jul 06 20:22:06.367712 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.367670    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"]
Jul 06 20:22:06.368246 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.367992    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerName="istio-proxy"
Jul 06 20:22:06.368246 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.368003    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerName="istio-proxy"
Jul 06 20:22:06.368246 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.368023    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3af568c4-4f75-4ef5-bb36-5aa3e6c65190" containerName="maas-api"
Jul 06 20:22:06.368246 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.368029    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3af568c4-4f75-4ef5-bb36-5aa3e6c65190" containerName="maas-api"
Jul 06 20:22:06.368246 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.368086    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerName="istio-proxy"
Jul 06 20:22:06.368246 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.368094    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3af568c4-4f75-4ef5-bb36-5aa3e6c65190" containerName="maas-api"
Jul 06 20:22:06.368246 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.368100    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerName="istio-proxy"
Jul 06 20:22:06.370983 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.370961    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.373686 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.373659    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-86976a02-openshift-default-dockercfg-7hxdg\""
Jul 06 20:22:06.373839 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.373760    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 20:22:06.389752 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.389719    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"]
Jul 06 20:22:06.400503 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.400472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-envoy\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.400672 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.400514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-credential-socket\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.400672 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.400541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-certs\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.400672 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.400557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-podinfo\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.400672 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.400586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0f1e210-af91-4212-8f19-4008c62d13ae-istiod-ca-cert\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.400906 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.400684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9wjf\" (UniqueName: \"kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-kube-api-access-j9wjf\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.400906 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.400717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-data\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.400906 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.400757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-token\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.400906 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.400775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-socket\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.501615 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.501579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-certs\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.501615 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.501619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-podinfo\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.501869 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.501650    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0f1e210-af91-4212-8f19-4008c62d13ae-istiod-ca-cert\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.501869 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.501697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9wjf\" (UniqueName: \"kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-kube-api-access-j9wjf\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.501869 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.501732    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-data\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.502078 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.501921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-token\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.502078 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.501973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-socket\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.502078 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.502017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-envoy\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.502078 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.502029    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-certs\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.502078 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.502062    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-credential-socket\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.502364 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.502093    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-data\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.502439 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.502417    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-socket\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.502498 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.502428    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-credential-socket\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.502555 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.502527    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0f1e210-af91-4212-8f19-4008c62d13ae-istiod-ca-cert\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.504060 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.504033    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-envoy\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.504366 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.504347    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-podinfo\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.509854 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.509831    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-token\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.509970 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.509911    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9wjf\" (UniqueName: \"kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-kube-api-access-j9wjf\") pod \"e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") " pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.681336 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.681291    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:06.821687 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.821660    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"]
Jul 06 20:22:06.824427 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:22:06.824383    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0f1e210_af91_4212_8f19_4008c62d13ae.slice/crio-3c2b597c60183143b5d95792d62e5cf3e087a7fc3eb7da7bf46ffca4395cb086 WatchSource:0}: Error finding container 3c2b597c60183143b5d95792d62e5cf3e087a7fc3eb7da7bf46ffca4395cb086: Status 404 returned error can't find the container with id 3c2b597c60183143b5d95792d62e5cf3e087a7fc3eb7da7bf46ffca4395cb086
Jul 06 20:22:06.826673 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.826643    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:22:06.826817 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.826705    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:22:06.826817 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.826737    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:22:06.979253 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.979100    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" event={"ID":"d0f1e210-af91-4212-8f19-4008c62d13ae","Type":"ContainerStarted","Data":"625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381"}
Jul 06 20:22:06.979253 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.979200    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" event={"ID":"d0f1e210-af91-4212-8f19-4008c62d13ae","Type":"ContainerStarted","Data":"3c2b597c60183143b5d95792d62e5cf3e087a7fc3eb7da7bf46ffca4395cb086"}
Jul 06 20:22:06.999969 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:06.999913    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" podStartSLOduration=0.999894152 podStartE2EDuration="999.894152ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:22:06.998821103 +0000 UTC m=+1907.310276326" watchObservedRunningTime="2026-07-06 20:22:06.999894152 +0000 UTC m=+1907.311349372"
Jul 06 20:22:07.682514 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:07.682476    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:07.687463 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:07.687433    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:07.984388 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:07.984301    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:07.985256 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:07.985236    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:08.983131 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:08.983094    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"]
Jul 06 20:22:08.983526 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:08.983443    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerName="istio-proxy"
Jul 06 20:22:08.983526 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:08.983455    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f5d57b4-6f22-4387-aad1-cfa60c21c745" containerName="istio-proxy"
Jul 06 20:22:08.985626 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:08.985601    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:08.988205 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:08.988163    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-86976a02-serving-cert\""
Jul 06 20:22:08.988416 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:08.988394    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mthr8\""
Jul 06 20:22:08.998438 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:08.998413    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"]
Jul 06 20:22:09.021241 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:09.021201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f72n\" (UniqueName: \"kubernetes.io/projected/3c79f811-9308-4f63-88e8-3495c359b3e6-kube-api-access-2f72n\") pod \"maas-api-e2e-mt-86976a02-66ddff797f-p2rz7\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") " pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:09.021429 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:09.021282    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c79f811-9308-4f63-88e8-3495c359b3e6-openshift-service-ca\") pod \"maas-api-e2e-mt-86976a02-66ddff797f-p2rz7\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") " pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:09.021429 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:09.021322    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c79f811-9308-4f63-88e8-3495c359b3e6-maas-api-tls\") pod \"maas-api-e2e-mt-86976a02-66ddff797f-p2rz7\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") " pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:09.122386 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:09.122337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c79f811-9308-4f63-88e8-3495c359b3e6-openshift-service-ca\") pod \"maas-api-e2e-mt-86976a02-66ddff797f-p2rz7\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") " pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:09.122584 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:09.122439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c79f811-9308-4f63-88e8-3495c359b3e6-maas-api-tls\") pod \"maas-api-e2e-mt-86976a02-66ddff797f-p2rz7\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") " pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:09.122584 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:09.122483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f72n\" (UniqueName: \"kubernetes.io/projected/3c79f811-9308-4f63-88e8-3495c359b3e6-kube-api-access-2f72n\") pod \"maas-api-e2e-mt-86976a02-66ddff797f-p2rz7\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") " pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:09.122709 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:22:09.122580    2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-86976a02-serving-cert: secret "maas-api-e2e-mt-86976a02-serving-cert" not found
Jul 06 20:22:09.122709 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:22:09.122656    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c79f811-9308-4f63-88e8-3495c359b3e6-maas-api-tls podName:3c79f811-9308-4f63-88e8-3495c359b3e6 nodeName:}" failed. No retries permitted until 2026-07-06 20:22:09.622631595 +0000 UTC m=+1909.934086799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c79f811-9308-4f63-88e8-3495c359b3e6-maas-api-tls") pod "maas-api-e2e-mt-86976a02-66ddff797f-p2rz7" (UID: "3c79f811-9308-4f63-88e8-3495c359b3e6") : secret "maas-api-e2e-mt-86976a02-serving-cert" not found
Jul 06 20:22:09.123337 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:09.123319    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c79f811-9308-4f63-88e8-3495c359b3e6-openshift-service-ca\") pod \"maas-api-e2e-mt-86976a02-66ddff797f-p2rz7\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") " pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:09.137095 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:09.137061    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f72n\" (UniqueName: \"kubernetes.io/projected/3c79f811-9308-4f63-88e8-3495c359b3e6-kube-api-access-2f72n\") pod \"maas-api-e2e-mt-86976a02-66ddff797f-p2rz7\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") " pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:09.625758 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:09.625720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c79f811-9308-4f63-88e8-3495c359b3e6-maas-api-tls\") pod \"maas-api-e2e-mt-86976a02-66ddff797f-p2rz7\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") " pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:09.628316 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:09.628290    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c79f811-9308-4f63-88e8-3495c359b3e6-maas-api-tls\") pod \"maas-api-e2e-mt-86976a02-66ddff797f-p2rz7\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") " pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:09.897326 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:09.897222    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:10.040361 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:10.040256    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"]
Jul 06 20:22:10.042889 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:22:10.042863    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c79f811_9308_4f63_88e8_3495c359b3e6.slice/crio-5804069c488939042abe1c70fde8c9c13a9af664350ed26973fd234d875c9b78 WatchSource:0}: Error finding container 5804069c488939042abe1c70fde8c9c13a9af664350ed26973fd234d875c9b78: Status 404 returned error can't find the container with id 5804069c488939042abe1c70fde8c9c13a9af664350ed26973fd234d875c9b78
Jul 06 20:22:11.000506 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:11.000467    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7" event={"ID":"3c79f811-9308-4f63-88e8-3495c359b3e6","Type":"ContainerStarted","Data":"a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862"}
Jul 06 20:22:11.000742 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:11.000539    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7" event={"ID":"3c79f811-9308-4f63-88e8-3495c359b3e6","Type":"ContainerStarted","Data":"5804069c488939042abe1c70fde8c9c13a9af664350ed26973fd234d875c9b78"}
Jul 06 20:22:11.000742 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:11.000566    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:11.040291 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:11.040238    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7" podStartSLOduration=2.726266228 podStartE2EDuration="3.040218866s" totalImagesPullingTime="313.952638ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:22:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:22:11.039782953 +0000 UTC m=+1911.351238173" watchObservedRunningTime="2026-07-06 20:22:11.040218866 +0000 UTC m=+1911.351674086"
Jul 06 20:22:17.014020 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:17.013976    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:20.054458 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:20.054419    2577 generic.go:356] "Generic (PLEG): container finished" podID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerID="625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381" exitCode=0
Jul 06 20:22:20.054905 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:20.054491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" event={"ID":"d0f1e210-af91-4212-8f19-4008c62d13ae","Type":"ContainerDied","Data":"625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381"}
Jul 06 20:22:20.054905 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:20.054887    2577 scope.go:117] "RemoveContainer" containerID="625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381"
Jul 06 20:22:20.055911 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:20.055882    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:22:20.056003 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:20.055954    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:22:20.056003 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:20.055986    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:22:21.060408 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:21.060377    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" event={"ID":"d0f1e210-af91-4212-8f19-4008c62d13ae","Type":"ContainerStarted","Data":"d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8"}
Jul 06 20:22:21.681561 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:21.681517    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:21.682629 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:21.682600    2577 patch_prober.go:28] interesting pod/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 06 20:22:21.682749 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:21.682652    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 06 20:22:22.681887 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:22.681841    2577 patch_prober.go:28] interesting pod/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 06 20:22:22.682335 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:22.681907    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 06 20:22:22.984926 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:22.984825    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:23.682395 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:23.682354    2577 patch_prober.go:28] interesting pod/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 06 20:22:23.682816 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:23.682432    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 06 20:22:24.681695 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:24.681653    2577 patch_prober.go:28] interesting pod/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 06 20:22:24.681896 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:24.681721    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 06 20:22:25.685967 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:25.685929    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:25.686851 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:25.686832    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:22:46.206692 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.206655    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"]
Jul 06 20:22:46.207224 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.206934    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7" podUID="3c79f811-9308-4f63-88e8-3495c359b3e6" containerName="maas-api" containerID="cri-o://a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862" gracePeriod=30
Jul 06 20:22:46.451750 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.451723    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:46.570834 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.570742    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2f72n\" (UniqueName: \"kubernetes.io/projected/3c79f811-9308-4f63-88e8-3495c359b3e6-kube-api-access-2f72n\") pod \"3c79f811-9308-4f63-88e8-3495c359b3e6\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") "
Jul 06 20:22:46.570834 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.570832    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c79f811-9308-4f63-88e8-3495c359b3e6-maas-api-tls\") pod \"3c79f811-9308-4f63-88e8-3495c359b3e6\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") "
Jul 06 20:22:46.571033 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.570872    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c79f811-9308-4f63-88e8-3495c359b3e6-openshift-service-ca\") pod \"3c79f811-9308-4f63-88e8-3495c359b3e6\" (UID: \"3c79f811-9308-4f63-88e8-3495c359b3e6\") "
Jul 06 20:22:46.571320 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.571285    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c79f811-9308-4f63-88e8-3495c359b3e6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c79f811-9308-4f63-88e8-3495c359b3e6" (UID: "3c79f811-9308-4f63-88e8-3495c359b3e6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:22:46.572871 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.572839    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c79f811-9308-4f63-88e8-3495c359b3e6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c79f811-9308-4f63-88e8-3495c359b3e6" (UID: "3c79f811-9308-4f63-88e8-3495c359b3e6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:22:46.572970 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.572875    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c79f811-9308-4f63-88e8-3495c359b3e6-kube-api-access-2f72n" (OuterVolumeSpecName: "kube-api-access-2f72n") pod "3c79f811-9308-4f63-88e8-3495c359b3e6" (UID: "3c79f811-9308-4f63-88e8-3495c359b3e6"). InnerVolumeSpecName "kube-api-access-2f72n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:22:46.671876 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.671835    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c79f811-9308-4f63-88e8-3495c359b3e6-maas-api-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:22:46.671876 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.671870    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c79f811-9308-4f63-88e8-3495c359b3e6-openshift-service-ca\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:22:46.671876 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:46.671881    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f72n\" (UniqueName: \"kubernetes.io/projected/3c79f811-9308-4f63-88e8-3495c359b3e6-kube-api-access-2f72n\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:22:47.179066 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:47.179030    2577 generic.go:356] "Generic (PLEG): container finished" podID="3c79f811-9308-4f63-88e8-3495c359b3e6" containerID="a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862" exitCode=0
Jul 06 20:22:47.179320 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:47.179073    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"
Jul 06 20:22:47.179320 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:47.179091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7" event={"ID":"3c79f811-9308-4f63-88e8-3495c359b3e6","Type":"ContainerDied","Data":"a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862"}
Jul 06 20:22:47.179320 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:47.179143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7" event={"ID":"3c79f811-9308-4f63-88e8-3495c359b3e6","Type":"ContainerDied","Data":"5804069c488939042abe1c70fde8c9c13a9af664350ed26973fd234d875c9b78"}
Jul 06 20:22:47.179320 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:47.179187    2577 scope.go:117] "RemoveContainer" containerID="a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862"
Jul 06 20:22:47.188539 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:47.188520    2577 scope.go:117] "RemoveContainer" containerID="a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862"
Jul 06 20:22:47.188816 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:22:47.188794    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862\": container with ID starting with a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862 not found: ID does not exist" containerID="a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862"
Jul 06 20:22:47.188881 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:47.188826    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862"} err="failed to get container status \"a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862\": rpc error: code = NotFound desc = could not find container \"a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862\": container with ID starting with a5e13c4c614f43010ecd6f5da723ac554c33fb9b63f5da0de8aacd5c385dc862 not found: ID does not exist"
Jul 06 20:22:47.199790 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:47.199764    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"]
Jul 06 20:22:47.203650 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:47.203622    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-86976a02-66ddff797f-p2rz7"]
Jul 06 20:22:48.337669 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:22:48.337639    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c79f811-9308-4f63-88e8-3495c359b3e6" path="/var/lib/kubelet/pods/3c79f811-9308-4f63-88e8-3495c359b3e6/volumes"
Jul 06 20:23:39.719742 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:39.719705    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"]
Jul 06 20:23:39.720213 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:39.719975    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy" containerID="cri-o://d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8" gracePeriod=30
Jul 06 20:23:40.689074 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:40.689033    2577 patch_prober.go:28] interesting pod/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 20:23:40.689276 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:40.689107    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 20:23:44.972254 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.972223    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:23:44.994648 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.993978    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-certs\") pod \"d0f1e210-af91-4212-8f19-4008c62d13ae\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") "
Jul 06 20:23:44.994648 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.994036    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-data\") pod \"d0f1e210-af91-4212-8f19-4008c62d13ae\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") "
Jul 06 20:23:44.994648 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.994125    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-credential-socket\") pod \"d0f1e210-af91-4212-8f19-4008c62d13ae\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") "
Jul 06 20:23:44.994648 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.994158    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-envoy\") pod \"d0f1e210-af91-4212-8f19-4008c62d13ae\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") "
Jul 06 20:23:44.994648 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.994203    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0f1e210-af91-4212-8f19-4008c62d13ae-istiod-ca-cert\") pod \"d0f1e210-af91-4212-8f19-4008c62d13ae\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") "
Jul 06 20:23:44.994648 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.994233    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-token\") pod \"d0f1e210-af91-4212-8f19-4008c62d13ae\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") "
Jul 06 20:23:44.994648 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.994275    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-podinfo\") pod \"d0f1e210-af91-4212-8f19-4008c62d13ae\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") "
Jul 06 20:23:44.994648 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.994304    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9wjf\" (UniqueName: \"kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-kube-api-access-j9wjf\") pod \"d0f1e210-af91-4212-8f19-4008c62d13ae\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") "
Jul 06 20:23:44.994648 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.994329    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-socket\") pod \"d0f1e210-af91-4212-8f19-4008c62d13ae\" (UID: \"d0f1e210-af91-4212-8f19-4008c62d13ae\") "
Jul 06 20:23:44.995920 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.994745    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0f1e210-af91-4212-8f19-4008c62d13ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d0f1e210-af91-4212-8f19-4008c62d13ae" (UID: "d0f1e210-af91-4212-8f19-4008c62d13ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:23:44.995920 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.994757    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d0f1e210-af91-4212-8f19-4008c62d13ae" (UID: "d0f1e210-af91-4212-8f19-4008c62d13ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:23:44.995920 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.994919    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d0f1e210-af91-4212-8f19-4008c62d13ae" (UID: "d0f1e210-af91-4212-8f19-4008c62d13ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:23:44.995920 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.995153    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "d0f1e210-af91-4212-8f19-4008c62d13ae" (UID: "d0f1e210-af91-4212-8f19-4008c62d13ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:23:44.995920 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.995447    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d0f1e210-af91-4212-8f19-4008c62d13ae" (UID: "d0f1e210-af91-4212-8f19-4008c62d13ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:23:44.997048 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.997017    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "d0f1e210-af91-4212-8f19-4008c62d13ae" (UID: "d0f1e210-af91-4212-8f19-4008c62d13ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:23:44.997298 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.997269    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d0f1e210-af91-4212-8f19-4008c62d13ae" (UID: "d0f1e210-af91-4212-8f19-4008c62d13ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:23:44.997395 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.997335    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-kube-api-access-j9wjf" (OuterVolumeSpecName: "kube-api-access-j9wjf") pod "d0f1e210-af91-4212-8f19-4008c62d13ae" (UID: "d0f1e210-af91-4212-8f19-4008c62d13ae"). InnerVolumeSpecName "kube-api-access-j9wjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:23:44.997455 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:44.997438    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d0f1e210-af91-4212-8f19-4008c62d13ae" (UID: "d0f1e210-af91-4212-8f19-4008c62d13ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 20:23:45.095342 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.095297    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-certs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:23:45.095342 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.095332    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-data\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:23:45.095342 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.095341    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-credential-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:23:45.095342 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.095351    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-envoy\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:23:45.095670 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.095361    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0f1e210-af91-4212-8f19-4008c62d13ae-istiod-ca-cert\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:23:45.095670 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.095368    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-token\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:23:45.095670 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.095376    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0f1e210-af91-4212-8f19-4008c62d13ae-istio-podinfo\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:23:45.095670 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.095384    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9wjf\" (UniqueName: \"kubernetes.io/projected/d0f1e210-af91-4212-8f19-4008c62d13ae-kube-api-access-j9wjf\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:23:45.095670 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.095393    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0f1e210-af91-4212-8f19-4008c62d13ae-workload-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:23:45.437739 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.437697    2577 generic.go:356] "Generic (PLEG): container finished" podID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerID="d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8" exitCode=0
Jul 06 20:23:45.437919 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.437754    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"
Jul 06 20:23:45.437919 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.437775    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" event={"ID":"d0f1e210-af91-4212-8f19-4008c62d13ae","Type":"ContainerDied","Data":"d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8"}
Jul 06 20:23:45.437919 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.437834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n" event={"ID":"d0f1e210-af91-4212-8f19-4008c62d13ae","Type":"ContainerDied","Data":"3c2b597c60183143b5d95792d62e5cf3e087a7fc3eb7da7bf46ffca4395cb086"}
Jul 06 20:23:45.437919 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.437856    2577 scope.go:117] "RemoveContainer" containerID="d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8"
Jul 06 20:23:45.446899 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.446881    2577 scope.go:117] "RemoveContainer" containerID="625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381"
Jul 06 20:23:45.454861 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.454837    2577 scope.go:117] "RemoveContainer" containerID="d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8"
Jul 06 20:23:45.455262 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:23:45.455165    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8\": container with ID starting with d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8 not found: ID does not exist" containerID="d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8"
Jul 06 20:23:45.455514 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.455277    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8"} err="failed to get container status \"d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8\": rpc error: code = NotFound desc = could not find container \"d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8\": container with ID starting with d22d3d3427394332dffb3092ac092099540f1bf72e1ed878f18e8134f9c5cfa8 not found: ID does not exist"
Jul 06 20:23:45.455514 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.455304    2577 scope.go:117] "RemoveContainer" containerID="625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381"
Jul 06 20:23:45.455679 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:23:45.455587    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381\": container with ID starting with 625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381 not found: ID does not exist" containerID="625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381"
Jul 06 20:23:45.455679 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.455610    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381"} err="failed to get container status \"625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381\": rpc error: code = NotFound desc = could not find container \"625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381\": container with ID starting with 625849b65063c0fd6fef629b98f983408dcd16fa79a26fda9151ab7ea0e6c381 not found: ID does not exist"
Jul 06 20:23:45.459621 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.459593    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"]
Jul 06 20:23:45.463808 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:45.463784    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-86976a02-openshift-default-696cc6c98c-jdq8n"]
Jul 06 20:23:46.337942 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:23:46.337909    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" path="/var/lib/kubelet/pods/d0f1e210-af91-4212-8f19-4008c62d13ae/volumes"
Jul 06 20:24:34.077602 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.077568    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"]
Jul 06 20:24:34.078057 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.077869    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy"
Jul 06 20:24:34.078057 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.077881    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy"
Jul 06 20:24:34.078057 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.077891    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c79f811-9308-4f63-88e8-3495c359b3e6" containerName="maas-api"
Jul 06 20:24:34.078057 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.077897    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c79f811-9308-4f63-88e8-3495c359b3e6" containerName="maas-api"
Jul 06 20:24:34.078057 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.077947    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy"
Jul 06 20:24:34.078057 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.077957    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c79f811-9308-4f63-88e8-3495c359b3e6" containerName="maas-api"
Jul 06 20:24:34.082158 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.082124    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.085418 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.085397    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 20:24:34.085665 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.085651    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-62ae7f-openshift-default-dockercfg-rmdfr\""
Jul 06 20:24:34.095019 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.094996    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"]
Jul 06 20:24:34.131392 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.131347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc7e76a-a27e-4126-9410-f5f8616605d0-istiod-ca-cert\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.131392 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.131390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-socket\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.131597 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.131455    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-certs\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.131597 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.131474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-envoy\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.131597 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.131500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-credential-socket\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.131597 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.131540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-podinfo\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.131597 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.131571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-data\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.131758 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.131606    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlhtb\" (UniqueName: \"kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-kube-api-access-vlhtb\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.131758 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.131627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-token\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.232362 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.232326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-token\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.232544 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.232375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc7e76a-a27e-4126-9410-f5f8616605d0-istiod-ca-cert\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.232544 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.232408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-socket\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.232544 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.232461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-certs\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.232544 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.232486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-envoy\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.232544 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.232513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-credential-socket\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.232544 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.232539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-podinfo\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.232852 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.232572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-data\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.232852 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.232605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlhtb\" (UniqueName: \"kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-kube-api-access-vlhtb\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.233042 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.232890    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-socket\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.233042 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.232944    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-certs\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.233153 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.233064    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-credential-socket\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.233153 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.233087    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-data\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.233270 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.233253    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc7e76a-a27e-4126-9410-f5f8616605d0-istiod-ca-cert\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.234783 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.234755    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-envoy\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.235047 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.235028    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-podinfo\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.241339 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.241311    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-token\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.241587 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.241562    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlhtb\" (UniqueName: \"kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-kube-api-access-vlhtb\") pod \"e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") " pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.392405 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.392375    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:34.535325 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.535215    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"]
Jul 06 20:24:34.537531 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:24:34.537505    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc7e76a_a27e_4126_9410_f5f8616605d0.slice/crio-2ce03427140ac8425bfe7b7966323e6f406f36e3e8b76e6e31a0ca186d5b6f4b WatchSource:0}: Error finding container 2ce03427140ac8425bfe7b7966323e6f406f36e3e8b76e6e31a0ca186d5b6f4b: Status 404 returned error can't find the container with id 2ce03427140ac8425bfe7b7966323e6f406f36e3e8b76e6e31a0ca186d5b6f4b
Jul 06 20:24:34.539570 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.539538    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:24:34.539660 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.539603    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:24:34.539660 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.539629    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:24:34.644816 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.644723    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" event={"ID":"8cc7e76a-a27e-4126-9410-f5f8616605d0","Type":"ContainerStarted","Data":"24e3678a6214ae51d5676edb828bfc4b7369d81931defaab671bf1f5f5ea23c4"}
Jul 06 20:24:34.644816 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.644794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" event={"ID":"8cc7e76a-a27e-4126-9410-f5f8616605d0","Type":"ContainerStarted","Data":"2ce03427140ac8425bfe7b7966323e6f406f36e3e8b76e6e31a0ca186d5b6f4b"}
Jul 06 20:24:34.671170 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:34.671091    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" podStartSLOduration=0.671068949 podStartE2EDuration="671.068949ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:24:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:34.669880104 +0000 UTC m=+2054.981335326" watchObservedRunningTime="2026-07-06 20:24:34.671068949 +0000 UTC m=+2054.982524174"
Jul 06 20:24:35.393253 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:35.393212    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:35.397945 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:35.397917    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:35.649907 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:35.649818    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:35.650919 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:35.650895    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:36.932330 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:36.932295    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"]
Jul 06 20:24:36.932797 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:36.932617    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy"
Jul 06 20:24:36.932797 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:36.932629    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy"
Jul 06 20:24:36.932797 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:36.932689    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0f1e210-af91-4212-8f19-4008c62d13ae" containerName="istio-proxy"
Jul 06 20:24:36.935783 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:36.935764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:36.938534 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:36.938508    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mthr8\""
Jul 06 20:24:36.938677 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:36.938515    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-62ae7f-serving-cert\""
Jul 06 20:24:36.946053 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:36.946029    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"]
Jul 06 20:24:36.958716 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:36.958682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dd86405-bae7-4593-a8a5-9a0b6dba4904-maas-api-tls\") pod \"maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") " pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:36.958856 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:36.958718    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k47dk\" (UniqueName: \"kubernetes.io/projected/8dd86405-bae7-4593-a8a5-9a0b6dba4904-kube-api-access-k47dk\") pod \"maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") " pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:36.958856 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:36.958758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd86405-bae7-4593-a8a5-9a0b6dba4904-openshift-service-ca\") pod \"maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") " pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:37.059452 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:37.059417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dd86405-bae7-4593-a8a5-9a0b6dba4904-maas-api-tls\") pod \"maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") " pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:37.059452 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:37.059459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k47dk\" (UniqueName: \"kubernetes.io/projected/8dd86405-bae7-4593-a8a5-9a0b6dba4904-kube-api-access-k47dk\") pod \"maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") " pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:37.059670 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:37.059497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd86405-bae7-4593-a8a5-9a0b6dba4904-openshift-service-ca\") pod \"maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") " pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:37.060128 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:37.060109    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd86405-bae7-4593-a8a5-9a0b6dba4904-openshift-service-ca\") pod \"maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") " pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:37.061984 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:37.061958    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dd86405-bae7-4593-a8a5-9a0b6dba4904-maas-api-tls\") pod \"maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") " pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:37.068966 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:37.068942    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k47dk\" (UniqueName: \"kubernetes.io/projected/8dd86405-bae7-4593-a8a5-9a0b6dba4904-kube-api-access-k47dk\") pod \"maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") " pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:37.247617 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:37.247535    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:37.399668 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:37.399564    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"]
Jul 06 20:24:37.402374 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:24:37.402345    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd86405_bae7_4593_a8a5_9a0b6dba4904.slice/crio-f6ee27ce334599b76e23b008a12bcec7d43b77b3353857df6a278896d17367cb WatchSource:0}: Error finding container f6ee27ce334599b76e23b008a12bcec7d43b77b3353857df6a278896d17367cb: Status 404 returned error can't find the container with id f6ee27ce334599b76e23b008a12bcec7d43b77b3353857df6a278896d17367cb
Jul 06 20:24:37.403741 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:37.403725    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:24:37.660373 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:37.660327    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5" event={"ID":"8dd86405-bae7-4593-a8a5-9a0b6dba4904","Type":"ContainerStarted","Data":"f6ee27ce334599b76e23b008a12bcec7d43b77b3353857df6a278896d17367cb"}
Jul 06 20:24:38.666605 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:38.666567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5" event={"ID":"8dd86405-bae7-4593-a8a5-9a0b6dba4904","Type":"ContainerStarted","Data":"14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac"}
Jul 06 20:24:38.667003 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:38.666700    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:38.685988 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:38.685938    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5" podStartSLOduration=2.300606267 podStartE2EDuration="2.685921168s" totalImagesPullingTime="385.314901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:38.68445128 +0000 UTC m=+2058.995906501" watchObservedRunningTime="2026-07-06 20:24:38.685921168 +0000 UTC m=+2058.997376389"
Jul 06 20:24:41.745351 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.745258    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"]
Jul 06 20:24:41.749260 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.749235    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.751832 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.751811    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-259a14-openshift-default-dockercfg-cn5vd\""
Jul 06 20:24:41.764675 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.764647    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"]
Jul 06 20:24:41.803243 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.803201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-data\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.803243 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.803246    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tf8r\" (UniqueName: \"kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-kube-api-access-6tf8r\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.803510 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.803277    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-socket\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.803510 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.803293    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-podinfo\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.803510 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.803318    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-envoy\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.803510 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.803342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-certs\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.803510 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.803414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-credential-socket\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.803510 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.803451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-token\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.803510 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.803498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istiod-ca-cert\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904245 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904201    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-data\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904457 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tf8r\" (UniqueName: \"kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-kube-api-access-6tf8r\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904457 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904401    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-socket\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904457 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-podinfo\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904457 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-envoy\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904679 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-certs\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904679 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-credential-socket\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904679 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-token\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904679 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904589    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-data\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904679 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istiod-ca-cert\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904956 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904845    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-certs\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.904956 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.904904    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-credential-socket\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.905073 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.905056    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-socket\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.905350 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.905328    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istiod-ca-cert\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.907079 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.907034    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-envoy\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.907392 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.907364    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-podinfo\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.918264 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.917827    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tf8r\" (UniqueName: \"kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-kube-api-access-6tf8r\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:41.930159 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:41.930117    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-token\") pod \"e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") " pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:42.060029 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:42.059936    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:42.233979 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:42.233897    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"]
Jul 06 20:24:42.241457 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:24:42.241424    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b7ed78_711c_4493_ac0d_2b8acd0405c6.slice/crio-f7e38b721a67412294ba1bd4601c36605cdd81d323700e62d2768ee6cbbf5d92 WatchSource:0}: Error finding container f7e38b721a67412294ba1bd4601c36605cdd81d323700e62d2768ee6cbbf5d92: Status 404 returned error can't find the container with id f7e38b721a67412294ba1bd4601c36605cdd81d323700e62d2768ee6cbbf5d92
Jul 06 20:24:42.245283 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:42.245251    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:24:42.245364 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:42.245319    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:24:42.245364 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:42.245349    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:24:42.683993 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:42.683956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" event={"ID":"24b7ed78-711c-4493-ac0d-2b8acd0405c6","Type":"ContainerStarted","Data":"66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114"}
Jul 06 20:24:42.684151 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:42.684019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" event={"ID":"24b7ed78-711c-4493-ac0d-2b8acd0405c6","Type":"ContainerStarted","Data":"f7e38b721a67412294ba1bd4601c36605cdd81d323700e62d2768ee6cbbf5d92"}
Jul 06 20:24:42.707483 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:42.707416    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" podStartSLOduration=1.7073955870000002 podStartE2EDuration="1.707395587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:24:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:42.704501838 +0000 UTC m=+2063.015957061" watchObservedRunningTime="2026-07-06 20:24:42.707395587 +0000 UTC m=+2063.018850811"
Jul 06 20:24:43.060589 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:43.060493    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:43.065659 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:43.065632    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:43.687825 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:43.687787    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:43.688961 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:43.688938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:24:44.676027 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:44.675995    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:24:44.827202 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:44.823702    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"]
Jul 06 20:24:44.838563 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:44.838528    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:44.845146 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:44.844204    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-259a14-serving-cert\""
Jul 06 20:24:44.845661 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:44.845630    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"]
Jul 06 20:24:44.946798 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:44.946706    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88ff4c68-841a-4b7b-b763-479ea106e936-openshift-service-ca\") pod \"maas-api-e2e-inf-b-259a14-6c45489855-2x5bz\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") " pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:44.946964 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:44.946796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88ff4c68-841a-4b7b-b763-479ea106e936-maas-api-tls\") pod \"maas-api-e2e-inf-b-259a14-6c45489855-2x5bz\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") " pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:44.946964 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:44.946854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-642l4\" (UniqueName: \"kubernetes.io/projected/88ff4c68-841a-4b7b-b763-479ea106e936-kube-api-access-642l4\") pod \"maas-api-e2e-inf-b-259a14-6c45489855-2x5bz\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") " pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:45.048383 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:45.048347    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-642l4\" (UniqueName: \"kubernetes.io/projected/88ff4c68-841a-4b7b-b763-479ea106e936-kube-api-access-642l4\") pod \"maas-api-e2e-inf-b-259a14-6c45489855-2x5bz\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") " pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:45.048579 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:45.048405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88ff4c68-841a-4b7b-b763-479ea106e936-openshift-service-ca\") pod \"maas-api-e2e-inf-b-259a14-6c45489855-2x5bz\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") " pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:45.048579 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:45.048454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88ff4c68-841a-4b7b-b763-479ea106e936-maas-api-tls\") pod \"maas-api-e2e-inf-b-259a14-6c45489855-2x5bz\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") " pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:45.048579 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:24:45.048555    2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-259a14-serving-cert: secret "maas-api-e2e-inf-b-259a14-serving-cert" not found
Jul 06 20:24:45.048746 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:24:45.048614    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ff4c68-841a-4b7b-b763-479ea106e936-maas-api-tls podName:88ff4c68-841a-4b7b-b763-479ea106e936 nodeName:}" failed. No retries permitted until 2026-07-06 20:24:45.548595255 +0000 UTC m=+2065.860050470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/88ff4c68-841a-4b7b-b763-479ea106e936-maas-api-tls") pod "maas-api-e2e-inf-b-259a14-6c45489855-2x5bz" (UID: "88ff4c68-841a-4b7b-b763-479ea106e936") : secret "maas-api-e2e-inf-b-259a14-serving-cert" not found
Jul 06 20:24:45.049126 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:45.049104    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88ff4c68-841a-4b7b-b763-479ea106e936-openshift-service-ca\") pod \"maas-api-e2e-inf-b-259a14-6c45489855-2x5bz\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") " pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:45.060305 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:45.060274    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-642l4\" (UniqueName: \"kubernetes.io/projected/88ff4c68-841a-4b7b-b763-479ea106e936-kube-api-access-642l4\") pod \"maas-api-e2e-inf-b-259a14-6c45489855-2x5bz\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") " pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:45.553140 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:45.553101    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88ff4c68-841a-4b7b-b763-479ea106e936-maas-api-tls\") pod \"maas-api-e2e-inf-b-259a14-6c45489855-2x5bz\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") " pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:45.555533 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:45.555508    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88ff4c68-841a-4b7b-b763-479ea106e936-maas-api-tls\") pod \"maas-api-e2e-inf-b-259a14-6c45489855-2x5bz\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") " pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:45.753691 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:45.753655    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:45.896854 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:45.896829    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"]
Jul 06 20:24:45.899140 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:24:45.899109    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88ff4c68_841a_4b7b_b763_479ea106e936.slice/crio-bffbc65091fa59ca1a285ea7e7668a1e483f6d067c008b057d9a2f9a86659f80 WatchSource:0}: Error finding container bffbc65091fa59ca1a285ea7e7668a1e483f6d067c008b057d9a2f9a86659f80: Status 404 returned error can't find the container with id bffbc65091fa59ca1a285ea7e7668a1e483f6d067c008b057d9a2f9a86659f80
Jul 06 20:24:46.701314 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:46.701189    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz" event={"ID":"88ff4c68-841a-4b7b-b763-479ea106e936","Type":"ContainerStarted","Data":"53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30"}
Jul 06 20:24:46.701314 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:46.701254    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz" event={"ID":"88ff4c68-841a-4b7b-b763-479ea106e936","Type":"ContainerStarted","Data":"bffbc65091fa59ca1a285ea7e7668a1e483f6d067c008b057d9a2f9a86659f80"}
Jul 06 20:24:46.701534 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:46.701382    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:46.722950 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:46.722897    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz" podStartSLOduration=2.200837693 podStartE2EDuration="2.722879257s" totalImagesPullingTime="522.041564ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:46.720939503 +0000 UTC m=+2067.032394725" watchObservedRunningTime="2026-07-06 20:24:46.722879257 +0000 UTC m=+2067.034334477"
Jul 06 20:24:48.711777 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:48.711685    2577 generic.go:356] "Generic (PLEG): container finished" podID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerID="24e3678a6214ae51d5676edb828bfc4b7369d81931defaab671bf1f5f5ea23c4" exitCode=0
Jul 06 20:24:48.711777 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:48.711758    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" event={"ID":"8cc7e76a-a27e-4126-9410-f5f8616605d0","Type":"ContainerDied","Data":"24e3678a6214ae51d5676edb828bfc4b7369d81931defaab671bf1f5f5ea23c4"}
Jul 06 20:24:48.712296 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:48.712138    2577 scope.go:117] "RemoveContainer" containerID="24e3678a6214ae51d5676edb828bfc4b7369d81931defaab671bf1f5f5ea23c4"
Jul 06 20:24:48.713213 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:48.713169    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:24:48.713313 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:48.713266    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:24:48.713378 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:48.713354    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:24:49.723722 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:49.722998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" event={"ID":"8cc7e76a-a27e-4126-9410-f5f8616605d0","Type":"ContainerStarted","Data":"038eb0fe44b2605f0aeb36bff6096ed9c7522676b068176ab06f0cb28b8775f1"}
Jul 06 20:24:50.392544 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:50.392515    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:50.393680 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:50.393654    2577 patch_prober.go:28] interesting pod/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 06 20:24:50.393795 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:50.393701    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 06 20:24:50.649820 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:50.649734    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:51.393398 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:51.393362    2577 patch_prober.go:28] interesting pod/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 06 20:24:51.393773 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:51.393413    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 06 20:24:52.393525 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:52.393486    2577 patch_prober.go:28] interesting pod/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 06 20:24:52.393927 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:52.393558    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 06 20:24:52.717887 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:52.717784    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:24:53.393306 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:53.393271    2577 patch_prober.go:28] interesting pod/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 06 20:24:53.393483 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:53.393338    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 06 20:24:54.397334 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:54.397302    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:54.398288 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:54.398266    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:24:59.430263 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:59.430127    2577 patch_prober.go:28] interesting pod/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": read tcp 10.134.0.2:41790->10.134.0.36:15021: read: connection reset by peer" start-of-body=
Jul 06 20:24:59.430263 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:59.430204    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": read tcp 10.134.0.2:41790->10.134.0.36:15021: read: connection reset by peer"
Jul 06 20:24:59.782486 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:59.782395    2577 generic.go:356] "Generic (PLEG): container finished" podID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerID="66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114" exitCode=0
Jul 06 20:24:59.782486 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:59.782436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" event={"ID":"24b7ed78-711c-4493-ac0d-2b8acd0405c6","Type":"ContainerDied","Data":"66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114"}
Jul 06 20:24:59.782830 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:59.782813    2577 scope.go:117] "RemoveContainer" containerID="66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114"
Jul 06 20:24:59.783810 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:59.783782    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:24:59.783904 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:59.783864    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:24:59.783965 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:24:59.783911    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:25:00.788874 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:00.788836    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" event={"ID":"24b7ed78-711c-4493-ac0d-2b8acd0405c6","Type":"ContainerStarted","Data":"063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378"}
Jul 06 20:25:01.060619 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:01.060531    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:25:01.060778 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:01.060756    2577 patch_prober.go:28] interesting pod/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 06 20:25:01.060834 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:01.060804    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 06 20:25:02.060912 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:02.060869    2577 patch_prober.go:28] interesting pod/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 06 20:25:02.061431 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:02.060952    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 06 20:25:03.061261 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:03.061225    2577 patch_prober.go:28] interesting pod/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 06 20:25:03.061659 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:03.061287    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 06 20:25:04.061042 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:04.061007    2577 patch_prober.go:28] interesting pod/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 06 20:25:04.061249 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:04.061061    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 06 20:25:05.064028 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:05.063988    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:25:05.064475 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:05.064347    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:25:05.065145 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:05.065129    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:25:54.521353 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.521319    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"]
Jul 06 20:25:54.522162 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.522114    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5" podUID="8dd86405-bae7-4593-a8a5-9a0b6dba4904" containerName="maas-api" containerID="cri-o://14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac" gracePeriod=30
Jul 06 20:25:54.671706 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.671665    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5" podUID="8dd86405-bae7-4593-a8a5-9a0b6dba4904" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.35:8443/health\": dial tcp 10.134.0.35:8443: connect: connection refused"
Jul 06 20:25:54.768142 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.768120    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:25:54.872533 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.872510    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k47dk\" (UniqueName: \"kubernetes.io/projected/8dd86405-bae7-4593-a8a5-9a0b6dba4904-kube-api-access-k47dk\") pod \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") "
Jul 06 20:25:54.872682 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.872563    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd86405-bae7-4593-a8a5-9a0b6dba4904-openshift-service-ca\") pod \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") "
Jul 06 20:25:54.872682 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.872617    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dd86405-bae7-4593-a8a5-9a0b6dba4904-maas-api-tls\") pod \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\" (UID: \"8dd86405-bae7-4593-a8a5-9a0b6dba4904\") "
Jul 06 20:25:54.873005 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.872983    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dd86405-bae7-4593-a8a5-9a0b6dba4904-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8dd86405-bae7-4593-a8a5-9a0b6dba4904" (UID: "8dd86405-bae7-4593-a8a5-9a0b6dba4904"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:25:54.874581 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.874561    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd86405-bae7-4593-a8a5-9a0b6dba4904-kube-api-access-k47dk" (OuterVolumeSpecName: "kube-api-access-k47dk") pod "8dd86405-bae7-4593-a8a5-9a0b6dba4904" (UID: "8dd86405-bae7-4593-a8a5-9a0b6dba4904"). InnerVolumeSpecName "kube-api-access-k47dk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:25:54.874693 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.874672    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd86405-bae7-4593-a8a5-9a0b6dba4904-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8dd86405-bae7-4593-a8a5-9a0b6dba4904" (UID: "8dd86405-bae7-4593-a8a5-9a0b6dba4904"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:25:54.973830 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.973791    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k47dk\" (UniqueName: \"kubernetes.io/projected/8dd86405-bae7-4593-a8a5-9a0b6dba4904-kube-api-access-k47dk\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:25:54.973830 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.973824    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd86405-bae7-4593-a8a5-9a0b6dba4904-openshift-service-ca\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:25:54.973830 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:54.973834    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dd86405-bae7-4593-a8a5-9a0b6dba4904-maas-api-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:25:55.034043 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:55.034014    2577 generic.go:356] "Generic (PLEG): container finished" podID="8dd86405-bae7-4593-a8a5-9a0b6dba4904" containerID="14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac" exitCode=0
Jul 06 20:25:55.034266 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:55.034052    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5" event={"ID":"8dd86405-bae7-4593-a8a5-9a0b6dba4904","Type":"ContainerDied","Data":"14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac"}
Jul 06 20:25:55.034266 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:55.034093    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5" event={"ID":"8dd86405-bae7-4593-a8a5-9a0b6dba4904","Type":"ContainerDied","Data":"f6ee27ce334599b76e23b008a12bcec7d43b77b3353857df6a278896d17367cb"}
Jul 06 20:25:55.034266 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:55.034114    2577 scope.go:117] "RemoveContainer" containerID="14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac"
Jul 06 20:25:55.034266 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:55.034055    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"
Jul 06 20:25:55.043300 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:55.043283    2577 scope.go:117] "RemoveContainer" containerID="14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac"
Jul 06 20:25:55.043549 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:25:55.043532    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac\": container with ID starting with 14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac not found: ID does not exist" containerID="14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac"
Jul 06 20:25:55.043593 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:55.043557    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac"} err="failed to get container status \"14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac\": rpc error: code = NotFound desc = could not find container \"14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac\": container with ID starting with 14a748fbc5fdcfc0fd183476d4ce38cb9fc998e54dbe84884f9980aef1b74dac not found: ID does not exist"
Jul 06 20:25:55.054431 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:55.054406    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"]
Jul 06 20:25:55.057984 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:55.057963    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-62ae7f-5dd9b7cb4f-w4xw5"]
Jul 06 20:25:56.338884 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:25:56.338838    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dd86405-bae7-4593-a8a5-9a0b6dba4904" path="/var/lib/kubelet/pods/8dd86405-bae7-4593-a8a5-9a0b6dba4904/volumes"
Jul 06 20:26:36.003121 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:36.003090    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"]
Jul 06 20:26:36.003646 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:36.003342    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy" containerID="cri-o://038eb0fe44b2605f0aeb36bff6096ed9c7522676b068176ab06f0cb28b8775f1" gracePeriod=30
Jul 06 20:26:37.140323 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.140274    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"]
Jul 06 20:26:37.140795 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.140523    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz" podUID="88ff4c68-841a-4b7b-b763-479ea106e936" containerName="maas-api" containerID="cri-o://53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30" gracePeriod=30
Jul 06 20:26:37.388564 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.388537    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:26:37.452161 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.452058    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-642l4\" (UniqueName: \"kubernetes.io/projected/88ff4c68-841a-4b7b-b763-479ea106e936-kube-api-access-642l4\") pod \"88ff4c68-841a-4b7b-b763-479ea106e936\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") "
Jul 06 20:26:37.452385 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.452169    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88ff4c68-841a-4b7b-b763-479ea106e936-maas-api-tls\") pod \"88ff4c68-841a-4b7b-b763-479ea106e936\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") "
Jul 06 20:26:37.452385 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.452243    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88ff4c68-841a-4b7b-b763-479ea106e936-openshift-service-ca\") pod \"88ff4c68-841a-4b7b-b763-479ea106e936\" (UID: \"88ff4c68-841a-4b7b-b763-479ea106e936\") "
Jul 06 20:26:37.452580 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.452550    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88ff4c68-841a-4b7b-b763-479ea106e936-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "88ff4c68-841a-4b7b-b763-479ea106e936" (UID: "88ff4c68-841a-4b7b-b763-479ea106e936"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:26:37.454221 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.454200    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88ff4c68-841a-4b7b-b763-479ea106e936-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "88ff4c68-841a-4b7b-b763-479ea106e936" (UID: "88ff4c68-841a-4b7b-b763-479ea106e936"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:26:37.454298 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.454232    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88ff4c68-841a-4b7b-b763-479ea106e936-kube-api-access-642l4" (OuterVolumeSpecName: "kube-api-access-642l4") pod "88ff4c68-841a-4b7b-b763-479ea106e936" (UID: "88ff4c68-841a-4b7b-b763-479ea106e936"). InnerVolumeSpecName "kube-api-access-642l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:26:37.552983 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.552943    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-642l4\" (UniqueName: \"kubernetes.io/projected/88ff4c68-841a-4b7b-b763-479ea106e936-kube-api-access-642l4\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:37.552983 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.552976    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88ff4c68-841a-4b7b-b763-479ea106e936-maas-api-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:37.552983 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:37.552985    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88ff4c68-841a-4b7b-b763-479ea106e936-openshift-service-ca\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:38.233030 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:38.232996    2577 generic.go:356] "Generic (PLEG): container finished" podID="88ff4c68-841a-4b7b-b763-479ea106e936" containerID="53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30" exitCode=0
Jul 06 20:26:38.233452 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:38.233038    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"
Jul 06 20:26:38.233452 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:38.233083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz" event={"ID":"88ff4c68-841a-4b7b-b763-479ea106e936","Type":"ContainerDied","Data":"53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30"}
Jul 06 20:26:38.233452 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:38.233134    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz" event={"ID":"88ff4c68-841a-4b7b-b763-479ea106e936","Type":"ContainerDied","Data":"bffbc65091fa59ca1a285ea7e7668a1e483f6d067c008b057d9a2f9a86659f80"}
Jul 06 20:26:38.233452 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:38.233159    2577 scope.go:117] "RemoveContainer" containerID="53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30"
Jul 06 20:26:38.242246 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:38.242226    2577 scope.go:117] "RemoveContainer" containerID="53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30"
Jul 06 20:26:38.242513 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:26:38.242495    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30\": container with ID starting with 53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30 not found: ID does not exist" containerID="53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30"
Jul 06 20:26:38.242563 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:38.242522    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30"} err="failed to get container status \"53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30\": rpc error: code = NotFound desc = could not find container \"53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30\": container with ID starting with 53f0d51f2b1ddb84780d25ac6933cd6241fd21a9a06046e546d0649958975b30 not found: ID does not exist"
Jul 06 20:26:38.254887 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:38.254864    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"]
Jul 06 20:26:38.258961 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:38.258940    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-259a14-6c45489855-2x5bz"]
Jul 06 20:26:38.343218 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:38.343166    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88ff4c68-841a-4b7b-b763-479ea106e936" path="/var/lib/kubelet/pods/88ff4c68-841a-4b7b-b763-479ea106e936/volumes"
Jul 06 20:26:39.399846 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:39.399813    2577 patch_prober.go:28] interesting pod/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 20:26:39.400268 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:39.399890    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 20:26:41.248828 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.248802    2577 generic.go:356] "Generic (PLEG): container finished" podID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerID="038eb0fe44b2605f0aeb36bff6096ed9c7522676b068176ab06f0cb28b8775f1" exitCode=0
Jul 06 20:26:41.249207 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.248871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" event={"ID":"8cc7e76a-a27e-4126-9410-f5f8616605d0","Type":"ContainerDied","Data":"038eb0fe44b2605f0aeb36bff6096ed9c7522676b068176ab06f0cb28b8775f1"}
Jul 06 20:26:41.249207 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.248924    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz" event={"ID":"8cc7e76a-a27e-4126-9410-f5f8616605d0","Type":"ContainerDied","Data":"2ce03427140ac8425bfe7b7966323e6f406f36e3e8b76e6e31a0ca186d5b6f4b"}
Jul 06 20:26:41.249207 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.248950    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ce03427140ac8425bfe7b7966323e6f406f36e3e8b76e6e31a0ca186d5b6f4b"
Jul 06 20:26:41.249207 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.248964    2577 scope.go:117] "RemoveContainer" containerID="24e3678a6214ae51d5676edb828bfc4b7369d81931defaab671bf1f5f5ea23c4"
Jul 06 20:26:41.251299 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.251282    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:26:41.282336 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282255    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-data\") pod \"8cc7e76a-a27e-4126-9410-f5f8616605d0\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") "
Jul 06 20:26:41.282336 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282288    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-token\") pod \"8cc7e76a-a27e-4126-9410-f5f8616605d0\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") "
Jul 06 20:26:41.282336 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282306    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlhtb\" (UniqueName: \"kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-kube-api-access-vlhtb\") pod \"8cc7e76a-a27e-4126-9410-f5f8616605d0\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") "
Jul 06 20:26:41.282336 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282332    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-credential-socket\") pod \"8cc7e76a-a27e-4126-9410-f5f8616605d0\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") "
Jul 06 20:26:41.282639 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282510    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc7e76a-a27e-4126-9410-f5f8616605d0-istiod-ca-cert\") pod \"8cc7e76a-a27e-4126-9410-f5f8616605d0\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") "
Jul 06 20:26:41.282639 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282588    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-socket\") pod \"8cc7e76a-a27e-4126-9410-f5f8616605d0\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") "
Jul 06 20:26:41.282639 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282611    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8cc7e76a-a27e-4126-9410-f5f8616605d0" (UID: "8cc7e76a-a27e-4126-9410-f5f8616605d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:41.282639 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282627    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-certs\") pod \"8cc7e76a-a27e-4126-9410-f5f8616605d0\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") "
Jul 06 20:26:41.282832 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282652    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-podinfo\") pod \"8cc7e76a-a27e-4126-9410-f5f8616605d0\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") "
Jul 06 20:26:41.282832 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282677    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-envoy\") pod \"8cc7e76a-a27e-4126-9410-f5f8616605d0\" (UID: \"8cc7e76a-a27e-4126-9410-f5f8616605d0\") "
Jul 06 20:26:41.282927 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282842    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8cc7e76a-a27e-4126-9410-f5f8616605d0" (UID: "8cc7e76a-a27e-4126-9410-f5f8616605d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:41.282927 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282851    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8cc7e76a-a27e-4126-9410-f5f8616605d0" (UID: "8cc7e76a-a27e-4126-9410-f5f8616605d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:41.282927 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282858    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cc7e76a-a27e-4126-9410-f5f8616605d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8cc7e76a-a27e-4126-9410-f5f8616605d0" (UID: "8cc7e76a-a27e-4126-9410-f5f8616605d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:26:41.283066 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.282997    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-credential-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:41.283066 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.283015    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc7e76a-a27e-4126-9410-f5f8616605d0-istiod-ca-cert\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:41.283066 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.283030    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:41.283066 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.283041    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-workload-certs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:41.283386 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.283365    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "8cc7e76a-a27e-4126-9410-f5f8616605d0" (UID: "8cc7e76a-a27e-4126-9410-f5f8616605d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:41.284967 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.284942    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-kube-api-access-vlhtb" (OuterVolumeSpecName: "kube-api-access-vlhtb") pod "8cc7e76a-a27e-4126-9410-f5f8616605d0" (UID: "8cc7e76a-a27e-4126-9410-f5f8616605d0"). InnerVolumeSpecName "kube-api-access-vlhtb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:26:41.285086 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.285069    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8cc7e76a-a27e-4126-9410-f5f8616605d0" (UID: "8cc7e76a-a27e-4126-9410-f5f8616605d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 20:26:41.285158 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.285091    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8cc7e76a-a27e-4126-9410-f5f8616605d0" (UID: "8cc7e76a-a27e-4126-9410-f5f8616605d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:41.285158 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.285094    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "8cc7e76a-a27e-4126-9410-f5f8616605d0" (UID: "8cc7e76a-a27e-4126-9410-f5f8616605d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:26:41.383565 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.383525    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-data\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:41.383565 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.383560    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-token\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:41.383565 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.383569    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlhtb\" (UniqueName: \"kubernetes.io/projected/8cc7e76a-a27e-4126-9410-f5f8616605d0-kube-api-access-vlhtb\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:41.383810 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.383578    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-podinfo\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:41.383810 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:41.383589    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc7e76a-a27e-4126-9410-f5f8616605d0-istio-envoy\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:42.254597 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:42.254566    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"
Jul 06 20:26:42.288371 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:42.288346    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"]
Jul 06 20:26:42.292592 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:42.292567    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-62ae7f-openshift-default-6c44b76795-vnbcz"]
Jul 06 20:26:42.337357 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:42.337330    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" path="/var/lib/kubelet/pods/8cc7e76a-a27e-4126-9410-f5f8616605d0/volumes"
Jul 06 20:26:42.821037 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:42.820995    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"]
Jul 06 20:26:42.821276 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:42.821251    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy" containerID="cri-o://063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378" gracePeriod=30
Jul 06 20:26:43.674939 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.674900    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"]
Jul 06 20:26:43.675426 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.675407    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd86405-bae7-4593-a8a5-9a0b6dba4904" containerName="maas-api"
Jul 06 20:26:43.675500 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.675428    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd86405-bae7-4593-a8a5-9a0b6dba4904" containerName="maas-api"
Jul 06 20:26:43.675500 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.675447    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy"
Jul 06 20:26:43.675500 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.675456    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy"
Jul 06 20:26:43.675500 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.675471    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88ff4c68-841a-4b7b-b763-479ea106e936" containerName="maas-api"
Jul 06 20:26:43.675500 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.675479    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88ff4c68-841a-4b7b-b763-479ea106e936" containerName="maas-api"
Jul 06 20:26:43.675500 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.675490    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy"
Jul 06 20:26:43.675500 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.675498    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy"
Jul 06 20:26:43.675812 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.675589    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88ff4c68-841a-4b7b-b763-479ea106e936" containerName="maas-api"
Jul 06 20:26:43.675812 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.675598    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy"
Jul 06 20:26:43.675812 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.675610    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd86405-bae7-4593-a8a5-9a0b6dba4904" containerName="maas-api"
Jul 06 20:26:43.679953 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.679934    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.682385 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.682367    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-83b464-openshift-default-dockercfg-sjt2r\""
Jul 06 20:26:43.690382 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.690361    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"]
Jul 06 20:26:43.700613 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.700592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-certs\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.700709 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.700633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-credential-socket\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.700709 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.700654    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-token\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.700803 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.700730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-data\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.700803 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.700778    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9mlb\" (UniqueName: \"kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-kube-api-access-b9mlb\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.700882 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.700812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-envoy\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.700882 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.700843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-socket\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.700978 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.700889    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-podinfo\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.700978 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.700959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a919a614-6880-4963-b6e9-0d7a1624bc26-istiod-ca-cert\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.801755 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.801722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-socket\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.801755 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.801756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-podinfo\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.801966 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.801859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a919a614-6880-4963-b6e9-0d7a1624bc26-istiod-ca-cert\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.801966 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.801883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-certs\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.801966 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.801918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-credential-socket\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.801966 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.801940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-token\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.802197 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.802127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-data\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.802267 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.802206    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-socket\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.802323 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.802277    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-certs\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.802379 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.802360    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-credential-socket\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.802439 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.802420    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-data\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.802526 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.802498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9mlb\" (UniqueName: \"kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-kube-api-access-b9mlb\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.802664 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.802555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-envoy\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.802735 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.802669    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a919a614-6880-4963-b6e9-0d7a1624bc26-istiod-ca-cert\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.804166 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.804141    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-podinfo\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.804483 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.804466    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-envoy\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.809746 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.809727    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-token\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.810206 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.810190    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9mlb\" (UniqueName: \"kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-kube-api-access-b9mlb\") pod \"e2e-api-a-83b464-openshift-default-56655978df-f8j6x\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") " pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:43.990651 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:43.990562    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:44.124738 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:44.124705    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"]
Jul 06 20:26:44.127966 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:26:44.127933    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda919a614_6880_4963_b6e9_0d7a1624bc26.slice/crio-b760f14dbccb5beb072335119e3e098b6fb57d547df287f51b95bf417ccf8aac WatchSource:0}: Error finding container b760f14dbccb5beb072335119e3e098b6fb57d547df287f51b95bf417ccf8aac: Status 404 returned error can't find the container with id b760f14dbccb5beb072335119e3e098b6fb57d547df287f51b95bf417ccf8aac
Jul 06 20:26:44.130203 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:44.130143    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:26:44.130278 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:44.130250    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:26:44.130314 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:44.130291    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:26:44.266915 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:44.266806    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" event={"ID":"a919a614-6880-4963-b6e9-0d7a1624bc26","Type":"ContainerStarted","Data":"4f3b0d28719e35d1d5357dabeb3ba4992ae11ccf5cf634131c9f15996cdac607"}
Jul 06 20:26:44.266915 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:44.266887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" event={"ID":"a919a614-6880-4963-b6e9-0d7a1624bc26","Type":"ContainerStarted","Data":"b760f14dbccb5beb072335119e3e098b6fb57d547df287f51b95bf417ccf8aac"}
Jul 06 20:26:44.286937 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:44.286884    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" podStartSLOduration=1.286867243 podStartE2EDuration="1.286867243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:26:44.28584713 +0000 UTC m=+2184.597302350" watchObservedRunningTime="2026-07-06 20:26:44.286867243 +0000 UTC m=+2184.598322465"
Jul 06 20:26:44.991713 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:44.991676    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:45.995787 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:45.995757    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:46.256768 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.256688    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"]
Jul 06 20:26:46.257352 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.257314    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cc7e76a-a27e-4126-9410-f5f8616605d0" containerName="istio-proxy"
Jul 06 20:26:46.260673 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.260651    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:46.263012 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.262989    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mthr8\""
Jul 06 20:26:46.263144 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.263093    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-83b464-serving-cert\""
Jul 06 20:26:46.274965 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.274944    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:46.275796 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.275777    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:26:46.280160 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.280138    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"]
Jul 06 20:26:46.326287 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.326256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-maas-api-tls\") pod \"maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") " pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:46.326453 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.326385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") " pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:46.326453 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.326413    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qmmp\" (UniqueName: \"kubernetes.io/projected/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-kube-api-access-5qmmp\") pod \"maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") " pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:46.426964 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.426931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-maas-api-tls\") pod \"maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") " pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:46.427138 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.426993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") " pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:46.427138 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.427044    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qmmp\" (UniqueName: \"kubernetes.io/projected/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-kube-api-access-5qmmp\") pod \"maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") " pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:46.427246 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:26:46.427230    2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-83b464-serving-cert: secret "maas-api-e2e-api-a-83b464-serving-cert" not found
Jul 06 20:26:46.427307 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:26:46.427295    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-maas-api-tls podName:1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c nodeName:}" failed. No retries permitted until 2026-07-06 20:26:46.927274329 +0000 UTC m=+2187.238729533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-maas-api-tls") pod "maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw" (UID: "1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c") : secret "maas-api-e2e-api-a-83b464-serving-cert" not found
Jul 06 20:26:46.427712 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.427694    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") " pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:46.452896 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.452869    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qmmp\" (UniqueName: \"kubernetes.io/projected/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-kube-api-access-5qmmp\") pod \"maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") " pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:46.931469 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.931438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-maas-api-tls\") pod \"maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") " pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:46.933696 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:46.933665    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-maas-api-tls\") pod \"maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") " pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:47.171991 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:47.171956    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:47.319925 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:47.319899    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"]
Jul 06 20:26:47.322615 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:26:47.322587    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cdc9274_fa35_416c_9e3c_c6e7df4ebb2c.slice/crio-0e2f43633ba841e9341fd34e4a071bc8f784842d39d3cc87d28da98844390e13 WatchSource:0}: Error finding container 0e2f43633ba841e9341fd34e4a071bc8f784842d39d3cc87d28da98844390e13: Status 404 returned error can't find the container with id 0e2f43633ba841e9341fd34e4a071bc8f784842d39d3cc87d28da98844390e13
Jul 06 20:26:48.068445 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.068423    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:26:48.142479 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.142447    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-credential-socket\") pod \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") "
Jul 06 20:26:48.142637 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.142491    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-envoy\") pod \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") "
Jul 06 20:26:48.142637 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.142546    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-data\") pod \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") "
Jul 06 20:26:48.142637 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.142566    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-podinfo\") pod \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") "
Jul 06 20:26:48.142637 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.142589    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tf8r\" (UniqueName: \"kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-kube-api-access-6tf8r\") pod \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") "
Jul 06 20:26:48.142841 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.142642    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-token\") pod \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") "
Jul 06 20:26:48.142841 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.142716    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "24b7ed78-711c-4493-ac0d-2b8acd0405c6" (UID: "24b7ed78-711c-4493-ac0d-2b8acd0405c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:48.142841 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.142766    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istiod-ca-cert\") pod \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") "
Jul 06 20:26:48.142841 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.142797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-certs\") pod \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") "
Jul 06 20:26:48.143094 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.142852    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-socket\") pod \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\" (UID: \"24b7ed78-711c-4493-ac0d-2b8acd0405c6\") "
Jul 06 20:26:48.143224 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.143204    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-credential-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:48.143224 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.143199    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "24b7ed78-711c-4493-ac0d-2b8acd0405c6" (UID: "24b7ed78-711c-4493-ac0d-2b8acd0405c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:26:48.143415 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.143396    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "24b7ed78-711c-4493-ac0d-2b8acd0405c6" (UID: "24b7ed78-711c-4493-ac0d-2b8acd0405c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:48.143472 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.143416    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "24b7ed78-711c-4493-ac0d-2b8acd0405c6" (UID: "24b7ed78-711c-4493-ac0d-2b8acd0405c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:48.143523 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.143487    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-data" (OuterVolumeSpecName: "istio-data") pod "24b7ed78-711c-4493-ac0d-2b8acd0405c6" (UID: "24b7ed78-711c-4493-ac0d-2b8acd0405c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:48.144853 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.144822    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "24b7ed78-711c-4493-ac0d-2b8acd0405c6" (UID: "24b7ed78-711c-4493-ac0d-2b8acd0405c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:26:48.144998 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.144978    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "24b7ed78-711c-4493-ac0d-2b8acd0405c6" (UID: "24b7ed78-711c-4493-ac0d-2b8acd0405c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 20:26:48.145136 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.145114    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-token" (OuterVolumeSpecName: "istio-token") pod "24b7ed78-711c-4493-ac0d-2b8acd0405c6" (UID: "24b7ed78-711c-4493-ac0d-2b8acd0405c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:26:48.145136 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.145123    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-kube-api-access-6tf8r" (OuterVolumeSpecName: "kube-api-access-6tf8r") pod "24b7ed78-711c-4493-ac0d-2b8acd0405c6" (UID: "24b7ed78-711c-4493-ac0d-2b8acd0405c6"). InnerVolumeSpecName "kube-api-access-6tf8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:26:48.244486 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.244449    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-data\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:48.244486 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.244481    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-podinfo\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:48.244486 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.244493    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tf8r\" (UniqueName: \"kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-kube-api-access-6tf8r\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:48.244923 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.244506    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-token\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:48.244923 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.244518    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istiod-ca-cert\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:48.244923 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.244529    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-certs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:48.244923 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.244539    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-workload-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:48.244923 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.244549    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24b7ed78-711c-4493-ac0d-2b8acd0405c6-istio-envoy\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:26:48.287308 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.287269    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw" event={"ID":"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c","Type":"ContainerStarted","Data":"04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a"}
Jul 06 20:26:48.287471 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.287334    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw" event={"ID":"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c","Type":"ContainerStarted","Data":"0e2f43633ba841e9341fd34e4a071bc8f784842d39d3cc87d28da98844390e13"}
Jul 06 20:26:48.287471 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.287359    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:48.288876 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.288853    2577 generic.go:356] "Generic (PLEG): container finished" podID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerID="063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378" exitCode=0
Jul 06 20:26:48.288990 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.288890    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"
Jul 06 20:26:48.288990 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.288934    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" event={"ID":"24b7ed78-711c-4493-ac0d-2b8acd0405c6","Type":"ContainerDied","Data":"063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378"}
Jul 06 20:26:48.288990 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.288982    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl" event={"ID":"24b7ed78-711c-4493-ac0d-2b8acd0405c6","Type":"ContainerDied","Data":"f7e38b721a67412294ba1bd4601c36605cdd81d323700e62d2768ee6cbbf5d92"}
Jul 06 20:26:48.289089 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.289005    2577 scope.go:117] "RemoveContainer" containerID="063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378"
Jul 06 20:26:48.297268 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.297252    2577 scope.go:117] "RemoveContainer" containerID="66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114"
Jul 06 20:26:48.304436 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.304421    2577 scope.go:117] "RemoveContainer" containerID="063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378"
Jul 06 20:26:48.304680 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:26:48.304663    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378\": container with ID starting with 063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378 not found: ID does not exist" containerID="063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378"
Jul 06 20:26:48.304717 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.304688    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378"} err="failed to get container status \"063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378\": rpc error: code = NotFound desc = could not find container \"063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378\": container with ID starting with 063356bbb594a46aa34640bfe761215fa2097a375224d3687f5c964326756378 not found: ID does not exist"
Jul 06 20:26:48.304717 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.304705    2577 scope.go:117] "RemoveContainer" containerID="66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114"
Jul 06 20:26:48.304946 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:26:48.304923    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114\": container with ID starting with 66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114 not found: ID does not exist" containerID="66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114"
Jul 06 20:26:48.305024 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.304949    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114"} err="failed to get container status \"66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114\": rpc error: code = NotFound desc = could not find container \"66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114\": container with ID starting with 66d226bb40102684cc364b4e605207421c3a5698bb73142ca5f0b30e3a628114 not found: ID does not exist"
Jul 06 20:26:48.311811 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.311772    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw" podStartSLOduration=1.8639094950000001 podStartE2EDuration="2.311759856s" totalImagesPullingTime="447.850361ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:26:48.310741996 +0000 UTC m=+2188.622197217" watchObservedRunningTime="2026-07-06 20:26:48.311759856 +0000 UTC m=+2188.623215076"
Jul 06 20:26:48.325917 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.325893    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"]
Jul 06 20:26:48.327750 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.327730    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-259a14-openshift-default-8858c8bc9-rtzwl"]
Jul 06 20:26:48.337530 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:48.337510    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" path="/var/lib/kubelet/pods/24b7ed78-711c-4493-ac0d-2b8acd0405c6/volumes"
Jul 06 20:26:51.376457 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.376424    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"]
Jul 06 20:26:51.376956 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.376734    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy"
Jul 06 20:26:51.376956 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.376749    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy"
Jul 06 20:26:51.376956 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.376760    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy"
Jul 06 20:26:51.376956 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.376765    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy"
Jul 06 20:26:51.376956 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.376824    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy"
Jul 06 20:26:51.380826 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.380805    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.383244 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.383226    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-650573-openshift-default-dockercfg-wnldn\""
Jul 06 20:26:51.396274 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.396251    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"]
Jul 06 20:26:51.474764 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.474729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e647436-ac15-4e56-a71e-31b822121170-istio-podinfo\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.474965 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.474784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q5lm\" (UniqueName: \"kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-kube-api-access-8q5lm\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.474965 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.474828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-envoy\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.474965 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.474846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-istio-token\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.474965 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.474919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-socket\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.474965 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.474941    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-certs\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.474965 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.474965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e647436-ac15-4e56-a71e-31b822121170-istiod-ca-cert\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.475268 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.474997    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-data\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.475268 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.475068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-credential-socket\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.576481 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.576449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e647436-ac15-4e56-a71e-31b822121170-istio-podinfo\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.576481 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.576486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5lm\" (UniqueName: \"kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-kube-api-access-8q5lm\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.576739 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.576510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-envoy\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.576739 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.576539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-istio-token\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.576739 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.576567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-socket\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.576739 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.576582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-certs\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.576739 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.576597    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e647436-ac15-4e56-a71e-31b822121170-istiod-ca-cert\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.576739 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.576618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-data\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.576739 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.576657    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-credential-socket\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.577087 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.576992    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-socket\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.577087 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.577050    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-data\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.577210 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.577096    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-certs\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.577210 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.577142    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-credential-socket\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.577637 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.577616    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e647436-ac15-4e56-a71e-31b822121170-istiod-ca-cert\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.578877 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.578849    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-envoy\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.579214 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.579193    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e647436-ac15-4e56-a71e-31b822121170-istio-podinfo\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.585151 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.585124    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-istio-token\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.585269 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.585251    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q5lm\" (UniqueName: \"kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-kube-api-access-8q5lm\") pod \"e2e-api-b-650573-openshift-default-9c49b6697-lk7hv\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") " pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.691613 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.691515    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:51.832793 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.832764    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"]
Jul 06 20:26:51.834958 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:26:51.834921    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e647436_ac15_4e56_a71e_31b822121170.slice/crio-6b27f8dab980ff85ee593bbfebc2167cd53dcf3ba999568159cd889c72ed9fe4 WatchSource:0}: Error finding container 6b27f8dab980ff85ee593bbfebc2167cd53dcf3ba999568159cd889c72ed9fe4: Status 404 returned error can't find the container with id 6b27f8dab980ff85ee593bbfebc2167cd53dcf3ba999568159cd889c72ed9fe4
Jul 06 20:26:51.837254 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.837213    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:26:51.837352 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.837306    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:26:51.837396 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:51.837350    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:26:52.318627 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:52.318593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" event={"ID":"2e647436-ac15-4e56-a71e-31b822121170","Type":"ContainerStarted","Data":"51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41"}
Jul 06 20:26:52.318804 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:52.318648    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" event={"ID":"2e647436-ac15-4e56-a71e-31b822121170","Type":"ContainerStarted","Data":"6b27f8dab980ff85ee593bbfebc2167cd53dcf3ba999568159cd889c72ed9fe4"}
Jul 06 20:26:52.350251 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:52.350201    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" podStartSLOduration=1.350185723 podStartE2EDuration="1.350185723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:26:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:26:52.348742459 +0000 UTC m=+2192.660197679" watchObservedRunningTime="2026-07-06 20:26:52.350185723 +0000 UTC m=+2192.661640933"
Jul 06 20:26:52.692497 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:52.692460    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:52.697151 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:52.697126    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:53.322839 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:53.322808    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:53.323629 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:53.323611    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:26:54.300829 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.300799    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:26:54.360850 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.360811    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"]
Jul 06 20:26:54.361329 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.361310    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b7ed78-711c-4493-ac0d-2b8acd0405c6" containerName="istio-proxy"
Jul 06 20:26:54.364286 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.364265    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:54.367626 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.367604    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-650573-serving-cert\""
Jul 06 20:26:54.387920 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.387892    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"]
Jul 06 20:26:54.402343 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.402305    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-openshift-service-ca\") pod \"maas-api-e2e-api-b-650573-7fc4b7878-w7v49\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") " pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:54.402497 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.402347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrcgv\" (UniqueName: \"kubernetes.io/projected/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-kube-api-access-jrcgv\") pod \"maas-api-e2e-api-b-650573-7fc4b7878-w7v49\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") " pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:54.402574 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.402556    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-maas-api-tls\") pod \"maas-api-e2e-api-b-650573-7fc4b7878-w7v49\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") " pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:54.503988 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.503951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-maas-api-tls\") pod \"maas-api-e2e-api-b-650573-7fc4b7878-w7v49\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") " pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:54.504224 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.504036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-openshift-service-ca\") pod \"maas-api-e2e-api-b-650573-7fc4b7878-w7v49\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") " pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:54.504224 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.504060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrcgv\" (UniqueName: \"kubernetes.io/projected/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-kube-api-access-jrcgv\") pod \"maas-api-e2e-api-b-650573-7fc4b7878-w7v49\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") " pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:54.504224 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:26:54.504099    2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-650573-serving-cert: secret "maas-api-e2e-api-b-650573-serving-cert" not found
Jul 06 20:26:54.504224 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:26:54.504199    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-maas-api-tls podName:1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5 nodeName:}" failed. No retries permitted until 2026-07-06 20:26:55.004156447 +0000 UTC m=+2195.315611645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-maas-api-tls") pod "maas-api-e2e-api-b-650573-7fc4b7878-w7v49" (UID: "1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5") : secret "maas-api-e2e-api-b-650573-serving-cert" not found
Jul 06 20:26:54.504673 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.504655    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-openshift-service-ca\") pod \"maas-api-e2e-api-b-650573-7fc4b7878-w7v49\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") " pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:54.512829 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:54.512809    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrcgv\" (UniqueName: \"kubernetes.io/projected/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-kube-api-access-jrcgv\") pod \"maas-api-e2e-api-b-650573-7fc4b7878-w7v49\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") " pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:55.008566 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:55.008537    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-maas-api-tls\") pod \"maas-api-e2e-api-b-650573-7fc4b7878-w7v49\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") " pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:55.011005 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:55.010974    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-maas-api-tls\") pod \"maas-api-e2e-api-b-650573-7fc4b7878-w7v49\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") " pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:55.276891 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:55.276808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:55.470802 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:55.470777    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"]
Jul 06 20:26:55.473222 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:26:55.473197    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4cbf04_7ee0_49d6_b1d3_681a70c79fa5.slice/crio-7db2e73473bb7c1426118d7afafa74761e255baf7ae90019880faa219413880f WatchSource:0}: Error finding container 7db2e73473bb7c1426118d7afafa74761e255baf7ae90019880faa219413880f: Status 404 returned error can't find the container with id 7db2e73473bb7c1426118d7afafa74761e255baf7ae90019880faa219413880f
Jul 06 20:26:56.351375 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:56.351286    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49" event={"ID":"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5","Type":"ContainerStarted","Data":"46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed"}
Jul 06 20:26:56.351375 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:56.351345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49" event={"ID":"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5","Type":"ContainerStarted","Data":"7db2e73473bb7c1426118d7afafa74761e255baf7ae90019880faa219413880f"}
Jul 06 20:26:56.351600 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:56.351406    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:26:56.373768 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:26:56.373676    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49" podStartSLOduration=1.9631245499999999 podStartE2EDuration="2.373654905s" totalImagesPullingTime="410.530355ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:26:56.372610315 +0000 UTC m=+2196.684065536" watchObservedRunningTime="2026-07-06 20:26:56.373654905 +0000 UTC m=+2196.685110125"
Jul 06 20:27:01.810360 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:01.810327    2577 patch_prober.go:28] interesting pod/e2e-api-a-83b464-openshift-default-56655978df-f8j6x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": read tcp 10.134.0.2:44828->10.134.0.38:15021: read: connection reset by peer" start-of-body=
Jul 06 20:27:01.810780 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:01.810396    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": read tcp 10.134.0.2:44828->10.134.0.38:15021: read: connection reset by peer"
Jul 06 20:27:02.361253 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:02.361226    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:27:02.385481 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:02.385448    2577 generic.go:356] "Generic (PLEG): container finished" podID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerID="4f3b0d28719e35d1d5357dabeb3ba4992ae11ccf5cf634131c9f15996cdac607" exitCode=0
Jul 06 20:27:02.385647 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:02.385488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" event={"ID":"a919a614-6880-4963-b6e9-0d7a1624bc26","Type":"ContainerDied","Data":"4f3b0d28719e35d1d5357dabeb3ba4992ae11ccf5cf634131c9f15996cdac607"}
Jul 06 20:27:02.385877 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:02.385862    2577 scope.go:117] "RemoveContainer" containerID="4f3b0d28719e35d1d5357dabeb3ba4992ae11ccf5cf634131c9f15996cdac607"
Jul 06 20:27:02.386903 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:02.386861    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:02.387037 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:02.386947    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:02.387037 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:02.387000    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:03.392389 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:03.392352    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" event={"ID":"a919a614-6880-4963-b6e9-0d7a1624bc26","Type":"ContainerStarted","Data":"937df6c95c31b48e0ae9e9754cf4d11fc235c52051172595764ad0fae673812f"}
Jul 06 20:27:03.991877 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:03.991833    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:27:03.992124 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:03.991902    2577 patch_prober.go:28] interesting pod/e2e-api-a-83b464-openshift-default-56655978df-f8j6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 06 20:27:03.992314 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:03.992289    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 06 20:27:04.991996 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:04.991955    2577 patch_prober.go:28] interesting pod/e2e-api-a-83b464-openshift-default-56655978df-f8j6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 06 20:27:04.992436 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:04.992027    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 06 20:27:05.991345 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:05.991305    2577 patch_prober.go:28] interesting pod/e2e-api-a-83b464-openshift-default-56655978df-f8j6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 06 20:27:05.991527 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:05.991369    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 06 20:27:06.991523 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:06.991483    2577 patch_prober.go:28] interesting pod/e2e-api-a-83b464-openshift-default-56655978df-f8j6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 06 20:27:06.991905 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:06.991539    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 06 20:27:07.648855 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:07.648818    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"]
Jul 06 20:27:07.649188 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:07.649141    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw" podUID="1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c" containerName="maas-api" containerID="cri-o://04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a" gracePeriod=30
Jul 06 20:27:07.915385 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:07.915352    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:27:07.995656 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:07.995622    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:27:07.996074 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:07.996024    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:27:07.996743 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:07.996722    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:27:08.067922 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.067284    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-maas-api-tls\") pod \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") "
Jul 06 20:27:08.067922 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.067332    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qmmp\" (UniqueName: \"kubernetes.io/projected/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-kube-api-access-5qmmp\") pod \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") "
Jul 06 20:27:08.068806 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.068208    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-openshift-service-ca\") pod \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\" (UID: \"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c\") "
Jul 06 20:27:08.069097 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.069050    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c" (UID: "1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:27:08.071730 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.071706    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c" (UID: "1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:27:08.072364 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.072219    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-kube-api-access-5qmmp" (OuterVolumeSpecName: "kube-api-access-5qmmp") pod "1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c" (UID: "1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c"). InnerVolumeSpecName "kube-api-access-5qmmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:27:08.169436 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.169394    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-maas-api-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:08.169436 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.169425    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qmmp\" (UniqueName: \"kubernetes.io/projected/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-kube-api-access-5qmmp\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:08.169436 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.169438    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c-openshift-service-ca\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:08.422367 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.422336    2577 generic.go:356] "Generic (PLEG): container finished" podID="1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c" containerID="04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a" exitCode=0
Jul 06 20:27:08.422570 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.422407    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"
Jul 06 20:27:08.422570 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.422421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw" event={"ID":"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c","Type":"ContainerDied","Data":"04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a"}
Jul 06 20:27:08.422570 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.422500    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw" event={"ID":"1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c","Type":"ContainerDied","Data":"0e2f43633ba841e9341fd34e4a071bc8f784842d39d3cc87d28da98844390e13"}
Jul 06 20:27:08.422570 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.422531    2577 scope.go:117] "RemoveContainer" containerID="04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a"
Jul 06 20:27:08.431492 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.431470    2577 scope.go:117] "RemoveContainer" containerID="04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a"
Jul 06 20:27:08.431745 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:27:08.431724    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a\": container with ID starting with 04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a not found: ID does not exist" containerID="04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a"
Jul 06 20:27:08.431826 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.431757    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a"} err="failed to get container status \"04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a\": rpc error: code = NotFound desc = could not find container \"04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a\": container with ID starting with 04b1ec7e14c1431e1f7cbc1c74a2f78dfb8c864c3056fba4e163660a092abe7a not found: ID does not exist"
Jul 06 20:27:08.438031 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.438010    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"]
Jul 06 20:27:08.444370 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:08.444347    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-83b464-7c76dbb5cb-2wjzw"]
Jul 06 20:27:09.363377 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:09.363332    2577 patch_prober.go:28] interesting pod/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 06 20:27:09.363773 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:09.363437    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 06 20:27:10.337385 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:10.337354    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c" path="/var/lib/kubelet/pods/1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c/volumes"
Jul 06 20:27:10.433973 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:10.433936    2577 generic.go:356] "Generic (PLEG): container finished" podID="2e647436-ac15-4e56-a71e-31b822121170" containerID="51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41" exitCode=0
Jul 06 20:27:10.434442 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:10.434016    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" event={"ID":"2e647436-ac15-4e56-a71e-31b822121170","Type":"ContainerDied","Data":"51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41"}
Jul 06 20:27:10.434553 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:10.434535    2577 scope.go:117] "RemoveContainer" containerID="51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41"
Jul 06 20:27:10.435732 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:10.435674    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:10.435827 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:10.435764    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:10.435827 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:10.435800    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:11.441307 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:11.441275    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" event={"ID":"2e647436-ac15-4e56-a71e-31b822121170","Type":"ContainerStarted","Data":"c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423"}
Jul 06 20:27:11.691858 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:11.691766    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:27:11.692037 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:11.691992    2577 patch_prober.go:28] interesting pod/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 06 20:27:11.692115 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:11.692053    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 06 20:27:12.692012 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:12.691967    2577 patch_prober.go:28] interesting pod/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 06 20:27:12.692467 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:12.692042    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 06 20:27:13.290463 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:13.290424    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"]
Jul 06 20:27:13.290718 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:13.290690    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy" containerID="cri-o://937df6c95c31b48e0ae9e9754cf4d11fc235c52051172595764ad0fae673812f" gracePeriod=30
Jul 06 20:27:13.692295 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:13.692261    2577 patch_prober.go:28] interesting pod/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 06 20:27:13.692703 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:13.692326    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 06 20:27:14.330705 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.327252    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"]
Jul 06 20:27:14.330705 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.327494    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49" podUID="1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5" containerName="maas-api" containerID="cri-o://46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed" gracePeriod=30
Jul 06 20:27:14.646512 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.646483    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:27:14.692750 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.692712    2577 patch_prober.go:28] interesting pod/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 06 20:27:14.693122 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.692781    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 06 20:27:14.730687 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.730624    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-maas-api-tls\") pod \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") "
Jul 06 20:27:14.731405 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.731377    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-openshift-service-ca\") pod \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") "
Jul 06 20:27:14.731657 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.731640    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrcgv\" (UniqueName: \"kubernetes.io/projected/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-kube-api-access-jrcgv\") pod \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\" (UID: \"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5\") "
Jul 06 20:27:14.733331 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.732533    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5" (UID: "1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:27:14.739352 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.739323    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5" (UID: "1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:27:14.743659 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.743640    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-kube-api-access-jrcgv" (OuterVolumeSpecName: "kube-api-access-jrcgv") pod "1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5" (UID: "1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5"). InnerVolumeSpecName "kube-api-access-jrcgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:27:14.832701 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.832648    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-maas-api-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:14.832701 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.832696    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-openshift-service-ca\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:14.832701 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:14.832711    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrcgv\" (UniqueName: \"kubernetes.io/projected/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5-kube-api-access-jrcgv\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:15.460241 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.460161    2577 generic.go:356] "Generic (PLEG): container finished" podID="1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5" containerID="46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed" exitCode=0
Jul 06 20:27:15.460241 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.460224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49" event={"ID":"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5","Type":"ContainerDied","Data":"46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed"}
Jul 06 20:27:15.460241 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.460236    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"
Jul 06 20:27:15.460511 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.460280    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49" event={"ID":"1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5","Type":"ContainerDied","Data":"7db2e73473bb7c1426118d7afafa74761e255baf7ae90019880faa219413880f"}
Jul 06 20:27:15.460511 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.460305    2577 scope.go:117] "RemoveContainer" containerID="46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed"
Jul 06 20:27:15.470084 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.470066    2577 scope.go:117] "RemoveContainer" containerID="46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed"
Jul 06 20:27:15.470365 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:27:15.470348    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed\": container with ID starting with 46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed not found: ID does not exist" containerID="46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed"
Jul 06 20:27:15.470431 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.470371    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed"} err="failed to get container status \"46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed\": rpc error: code = NotFound desc = could not find container \"46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed\": container with ID starting with 46d9b822efecec8a2af8dcc3de98737c13a42f6aba2bb166228581b97eab66ed not found: ID does not exist"
Jul 06 20:27:15.480612 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.480588    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"]
Jul 06 20:27:15.483727 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.483704    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-650573-7fc4b7878-w7v49"]
Jul 06 20:27:15.696248 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.696218    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:27:15.696627 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.696577    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:27:15.697339 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:15.697319    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:27:16.337988 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:16.337954    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5" path="/var/lib/kubelet/pods/1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5/volumes"
Jul 06 20:27:18.478405 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.478370    2577 generic.go:356] "Generic (PLEG): container finished" podID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerID="937df6c95c31b48e0ae9e9754cf4d11fc235c52051172595764ad0fae673812f" exitCode=0
Jul 06 20:27:18.478833 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.478444    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" event={"ID":"a919a614-6880-4963-b6e9-0d7a1624bc26","Type":"ContainerDied","Data":"937df6c95c31b48e0ae9e9754cf4d11fc235c52051172595764ad0fae673812f"}
Jul 06 20:27:18.478833 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.478528    2577 scope.go:117] "RemoveContainer" containerID="4f3b0d28719e35d1d5357dabeb3ba4992ae11ccf5cf634131c9f15996cdac607"
Jul 06 20:27:18.543759 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.543737    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:27:18.567660 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.567577    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9mlb\" (UniqueName: \"kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-kube-api-access-b9mlb\") pod \"a919a614-6880-4963-b6e9-0d7a1624bc26\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") "
Jul 06 20:27:18.567660 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.567627    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-certs\") pod \"a919a614-6880-4963-b6e9-0d7a1624bc26\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") "
Jul 06 20:27:18.567854 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.567761    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-socket\") pod \"a919a614-6880-4963-b6e9-0d7a1624bc26\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") "
Jul 06 20:27:18.567854 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.567828    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-data\") pod \"a919a614-6880-4963-b6e9-0d7a1624bc26\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") "
Jul 06 20:27:18.567939 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.567855    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-token\") pod \"a919a614-6880-4963-b6e9-0d7a1624bc26\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") "
Jul 06 20:27:18.567939 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.567906    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-credential-socket\") pod \"a919a614-6880-4963-b6e9-0d7a1624bc26\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") "
Jul 06 20:27:18.567939 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.567909    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a919a614-6880-4963-b6e9-0d7a1624bc26" (UID: "a919a614-6880-4963-b6e9-0d7a1624bc26"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:18.567939 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.567933    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-podinfo\") pod \"a919a614-6880-4963-b6e9-0d7a1624bc26\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") "
Jul 06 20:27:18.568141 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.567980    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-envoy\") pod \"a919a614-6880-4963-b6e9-0d7a1624bc26\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") "
Jul 06 20:27:18.568141 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.568004    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a919a614-6880-4963-b6e9-0d7a1624bc26-istiod-ca-cert\") pod \"a919a614-6880-4963-b6e9-0d7a1624bc26\" (UID: \"a919a614-6880-4963-b6e9-0d7a1624bc26\") "
Jul 06 20:27:18.568141 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.568020    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a919a614-6880-4963-b6e9-0d7a1624bc26" (UID: "a919a614-6880-4963-b6e9-0d7a1624bc26"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:18.568371 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.568319    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-certs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:18.568371 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.568339    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-workload-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:18.568694 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.568569    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a919a614-6880-4963-b6e9-0d7a1624bc26-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a919a614-6880-4963-b6e9-0d7a1624bc26" (UID: "a919a614-6880-4963-b6e9-0d7a1624bc26"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:27:18.568837 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.568764    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a919a614-6880-4963-b6e9-0d7a1624bc26" (UID: "a919a614-6880-4963-b6e9-0d7a1624bc26"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:18.568837 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.568800    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-data" (OuterVolumeSpecName: "istio-data") pod "a919a614-6880-4963-b6e9-0d7a1624bc26" (UID: "a919a614-6880-4963-b6e9-0d7a1624bc26"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:18.571240 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.571212    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a919a614-6880-4963-b6e9-0d7a1624bc26" (UID: "a919a614-6880-4963-b6e9-0d7a1624bc26"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 20:27:18.571789 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.571706    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a919a614-6880-4963-b6e9-0d7a1624bc26" (UID: "a919a614-6880-4963-b6e9-0d7a1624bc26"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:18.571789 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.571723    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-kube-api-access-b9mlb" (OuterVolumeSpecName: "kube-api-access-b9mlb") pod "a919a614-6880-4963-b6e9-0d7a1624bc26" (UID: "a919a614-6880-4963-b6e9-0d7a1624bc26"). InnerVolumeSpecName "kube-api-access-b9mlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:27:18.572131 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.572095    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-token" (OuterVolumeSpecName: "istio-token") pod "a919a614-6880-4963-b6e9-0d7a1624bc26" (UID: "a919a614-6880-4963-b6e9-0d7a1624bc26"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:27:18.669279 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.669242    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-credential-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:18.669279 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.669271    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-podinfo\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:18.669279 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.669280    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-envoy\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:18.669279 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.669288    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a919a614-6880-4963-b6e9-0d7a1624bc26-istiod-ca-cert\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:18.669548 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.669296    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9mlb\" (UniqueName: \"kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-kube-api-access-b9mlb\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:18.669548 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.669305    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-data\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:18.669548 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:18.669314    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a919a614-6880-4963-b6e9-0d7a1624bc26-istio-token\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:19.484378 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:19.484342    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x" event={"ID":"a919a614-6880-4963-b6e9-0d7a1624bc26","Type":"ContainerDied","Data":"b760f14dbccb5beb072335119e3e098b6fb57d547df287f51b95bf417ccf8aac"}
Jul 06 20:27:19.484378 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:19.484375    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"
Jul 06 20:27:19.484833 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:19.484419    2577 scope.go:117] "RemoveContainer" containerID="937df6c95c31b48e0ae9e9754cf4d11fc235c52051172595764ad0fae673812f"
Jul 06 20:27:19.504810 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:19.504781    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"]
Jul 06 20:27:19.507292 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:19.507265    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-83b464-openshift-default-56655978df-f8j6x"]
Jul 06 20:27:20.072518 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.072485    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"]
Jul 06 20:27:20.072754 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.072729    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy" containerID="cri-o://c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423" gracePeriod=30
Jul 06 20:27:20.338941 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.338868    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" path="/var/lib/kubelet/pods/a919a614-6880-4963-b6e9-0d7a1624bc26/volumes"
Jul 06 20:27:20.928303 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928261    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"]
Jul 06 20:27:20.928735 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928720    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c" containerName="maas-api"
Jul 06 20:27:20.928794 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928739    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c" containerName="maas-api"
Jul 06 20:27:20.928794 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928768    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy"
Jul 06 20:27:20.928794 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928777    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy"
Jul 06 20:27:20.928794 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928786    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy"
Jul 06 20:27:20.928922 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928795    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy"
Jul 06 20:27:20.928922 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928811    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5" containerName="maas-api"
Jul 06 20:27:20.928922 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928820    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5" containerName="maas-api"
Jul 06 20:27:20.928922 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928897    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy"
Jul 06 20:27:20.928922 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928908    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b4cbf04-7ee0-49d6-b1d3-681a70c79fa5" containerName="maas-api"
Jul 06 20:27:20.928922 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928920    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a919a614-6880-4963-b6e9-0d7a1624bc26" containerName="istio-proxy"
Jul 06 20:27:20.929123 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.928929    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cdc9274-fa35-416c-9e3c-c6e7df4ebb2c" containerName="maas-api"
Jul 06 20:27:20.932723 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.932701    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:20.935214 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.935166    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-482add-openshift-default-dockercfg-cwss2\""
Jul 06 20:27:20.944920 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.944895    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"]
Jul 06 20:27:20.991414 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.991383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-data\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:20.991608 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.991421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-envoy\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:20.991608 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.991459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mshr\" (UniqueName: \"kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-kube-api-access-5mshr\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:20.991608 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.991505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-certs\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:20.991608 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.991527    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-credential-socket\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:20.991608 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.991541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-podinfo\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:20.991836 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.991608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-token\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:20.991836 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.991641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istiod-ca-cert\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:20.991836 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:20.991687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-socket\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.092825 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.092788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-socket\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093013 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.092835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-data\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093013 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.092859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-envoy\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093013 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.092890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mshr\" (UniqueName: \"kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-kube-api-access-5mshr\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093013 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.092916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-certs\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093013 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.092935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-credential-socket\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093312 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.093049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-podinfo\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093312 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.093096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-token\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093312 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.093126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istiod-ca-cert\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093312 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.093192    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-socket\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093312 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.093257    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-data\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093569 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.093314    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-certs\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093569 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.093479    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-credential-socket\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.093801 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.093784    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istiod-ca-cert\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.095263 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.095239    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-envoy\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.095337 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.095286    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-podinfo\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.102965 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.102942    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mshr\" (UniqueName: \"kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-kube-api-access-5mshr\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.103234 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.103214    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-token\") pod \"e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") " pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.243217 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.243112    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:21.378969 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.378945    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"]
Jul 06 20:27:21.381378 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:27:21.381351    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda01634cc_f054_4811_b4a8_cc1b8bdd2120.slice/crio-4663831e3ad084368ed3cee7d5ad563167d09244fb0ab68de6aca99d83924983 WatchSource:0}: Error finding container 4663831e3ad084368ed3cee7d5ad563167d09244fb0ab68de6aca99d83924983: Status 404 returned error can't find the container with id 4663831e3ad084368ed3cee7d5ad563167d09244fb0ab68de6aca99d83924983
Jul 06 20:27:21.383509 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.383477    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:21.383609 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.383569    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:21.383672 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.383615    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:21.497739 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.497633    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" event={"ID":"a01634cc-f054-4811-b4a8-cc1b8bdd2120","Type":"ContainerStarted","Data":"c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856"}
Jul 06 20:27:21.497739 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.497704    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" event={"ID":"a01634cc-f054-4811-b4a8-cc1b8bdd2120","Type":"ContainerStarted","Data":"4663831e3ad084368ed3cee7d5ad563167d09244fb0ab68de6aca99d83924983"}
Jul 06 20:27:21.521320 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:21.521255    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" podStartSLOduration=1.521232519 podStartE2EDuration="1.521232519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:27:21.520951491 +0000 UTC m=+2221.832406703" watchObservedRunningTime="2026-07-06 20:27:21.521232519 +0000 UTC m=+2221.832687737"
Jul 06 20:27:22.243997 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:22.243952    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:22.248823 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:22.248798    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:22.501888 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:22.501802    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:22.502797 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:22.502775    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:25.348648 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.348623    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:27:25.434808 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.434773    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e647436-ac15-4e56-a71e-31b822121170-istio-podinfo\") pod \"2e647436-ac15-4e56-a71e-31b822121170\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") "
Jul 06 20:27:25.434808 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.434814    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-envoy\") pod \"2e647436-ac15-4e56-a71e-31b822121170\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") "
Jul 06 20:27:25.435071 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.434834    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-credential-socket\") pod \"2e647436-ac15-4e56-a71e-31b822121170\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") "
Jul 06 20:27:25.435071 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.434898    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-socket\") pod \"2e647436-ac15-4e56-a71e-31b822121170\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") "
Jul 06 20:27:25.435071 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.434954    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e647436-ac15-4e56-a71e-31b822121170-istiod-ca-cert\") pod \"2e647436-ac15-4e56-a71e-31b822121170\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") "
Jul 06 20:27:25.435071 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.435017    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-istio-token\") pod \"2e647436-ac15-4e56-a71e-31b822121170\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") "
Jul 06 20:27:25.435071 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.435040    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-certs\") pod \"2e647436-ac15-4e56-a71e-31b822121170\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") "
Jul 06 20:27:25.435071 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.435071    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8q5lm\" (UniqueName: \"kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-kube-api-access-8q5lm\") pod \"2e647436-ac15-4e56-a71e-31b822121170\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") "
Jul 06 20:27:25.435490 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.435113    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-data\") pod \"2e647436-ac15-4e56-a71e-31b822121170\" (UID: \"2e647436-ac15-4e56-a71e-31b822121170\") "
Jul 06 20:27:25.435490 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.435141    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2e647436-ac15-4e56-a71e-31b822121170" (UID: "2e647436-ac15-4e56-a71e-31b822121170"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:25.435490 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.435272    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2e647436-ac15-4e56-a71e-31b822121170" (UID: "2e647436-ac15-4e56-a71e-31b822121170"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:25.435490 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.435344    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2e647436-ac15-4e56-a71e-31b822121170" (UID: "2e647436-ac15-4e56-a71e-31b822121170"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:25.435490 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.435456    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e647436-ac15-4e56-a71e-31b822121170-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2e647436-ac15-4e56-a71e-31b822121170" (UID: "2e647436-ac15-4e56-a71e-31b822121170"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:27:25.435757 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.435513    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-certs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:25.435757 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.435530    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-credential-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:25.435757 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.435544    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-workload-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:25.436240 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.436214    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-data" (OuterVolumeSpecName: "istio-data") pod "2e647436-ac15-4e56-a71e-31b822121170" (UID: "2e647436-ac15-4e56-a71e-31b822121170"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:25.437226 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.437201    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e647436-ac15-4e56-a71e-31b822121170-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2e647436-ac15-4e56-a71e-31b822121170" (UID: "2e647436-ac15-4e56-a71e-31b822121170"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 20:27:25.437329 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.437238    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2e647436-ac15-4e56-a71e-31b822121170" (UID: "2e647436-ac15-4e56-a71e-31b822121170"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:27:25.437377 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.437326    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-kube-api-access-8q5lm" (OuterVolumeSpecName: "kube-api-access-8q5lm") pod "2e647436-ac15-4e56-a71e-31b822121170" (UID: "2e647436-ac15-4e56-a71e-31b822121170"). InnerVolumeSpecName "kube-api-access-8q5lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:27:25.437377 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.437340    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-istio-token" (OuterVolumeSpecName: "istio-token") pod "2e647436-ac15-4e56-a71e-31b822121170" (UID: "2e647436-ac15-4e56-a71e-31b822121170"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:27:25.519482 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.519448    2577 generic.go:356] "Generic (PLEG): container finished" podID="2e647436-ac15-4e56-a71e-31b822121170" containerID="c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423" exitCode=0
Jul 06 20:27:25.519657 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.519501    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"
Jul 06 20:27:25.519657 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.519506    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" event={"ID":"2e647436-ac15-4e56-a71e-31b822121170","Type":"ContainerDied","Data":"c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423"}
Jul 06 20:27:25.519657 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.519555    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv" event={"ID":"2e647436-ac15-4e56-a71e-31b822121170","Type":"ContainerDied","Data":"6b27f8dab980ff85ee593bbfebc2167cd53dcf3ba999568159cd889c72ed9fe4"}
Jul 06 20:27:25.519657 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.519581    2577 scope.go:117] "RemoveContainer" containerID="c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423"
Jul 06 20:27:25.528703 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.528686    2577 scope.go:117] "RemoveContainer" containerID="51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41"
Jul 06 20:27:25.535941 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.535926    2577 scope.go:117] "RemoveContainer" containerID="c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423"
Jul 06 20:27:25.536240 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:27:25.536218    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423\": container with ID starting with c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423 not found: ID does not exist" containerID="c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423"
Jul 06 20:27:25.536335 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.536246    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423"} err="failed to get container status \"c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423\": rpc error: code = NotFound desc = could not find container \"c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423\": container with ID starting with c8ce054149471389da52f413f947684e4e8215fc06a2bca7bb6e10bc28cbc423 not found: ID does not exist"
Jul 06 20:27:25.536335 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.536264    2577 scope.go:117] "RemoveContainer" containerID="51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41"
Jul 06 20:27:25.536335 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.536280    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e647436-ac15-4e56-a71e-31b822121170-istiod-ca-cert\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:25.536335 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.536299    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-istio-token\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:25.536335 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.536311    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8q5lm\" (UniqueName: \"kubernetes.io/projected/2e647436-ac15-4e56-a71e-31b822121170-kube-api-access-8q5lm\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:25.536335 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.536320    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-data\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:25.536335 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.536332    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e647436-ac15-4e56-a71e-31b822121170-istio-podinfo\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:25.536603 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.536343    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e647436-ac15-4e56-a71e-31b822121170-istio-envoy\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:27:25.536603 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:27:25.536470    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41\": container with ID starting with 51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41 not found: ID does not exist" containerID="51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41"
Jul 06 20:27:25.536603 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.536492    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41"} err="failed to get container status \"51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41\": rpc error: code = NotFound desc = could not find container \"51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41\": container with ID starting with 51f4696567fb1d155c5b1496a60f8d8fa36c67388c9e96c888bcbf399941ba41 not found: ID does not exist"
Jul 06 20:27:25.540762 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.540741    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"]
Jul 06 20:27:25.546716 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:25.546696    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-650573-openshift-default-9c49b6697-lk7hv"]
Jul 06 20:27:26.337117 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:26.337081    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e647436-ac15-4e56-a71e-31b822121170" path="/var/lib/kubelet/pods/2e647436-ac15-4e56-a71e-31b822121170/volumes"
Jul 06 20:27:28.603925 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.603890    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"]
Jul 06 20:27:28.604340 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.604282    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy"
Jul 06 20:27:28.604340 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.604298    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy"
Jul 06 20:27:28.604340 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.604312    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy"
Jul 06 20:27:28.604340 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.604319    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy"
Jul 06 20:27:28.604490 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.604370    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy"
Jul 06 20:27:28.608696 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.608674    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.612600 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.612572    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-45b15a-openshift-default-dockercfg-fgt6w\""
Jul 06 20:27:28.642521 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.642496    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"]
Jul 06 20:27:28.664367 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.664342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bbfd53-33e2-4320-a5ed-2dac82182143-istiod-ca-cert\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.664531 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.664374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-token\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.664531 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.664402    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zwbk\" (UniqueName: \"kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-kube-api-access-4zwbk\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.664531 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.664422    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-socket\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.664531 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.664441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-data\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.664531 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.664513    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-credential-socket\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.664785 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.664572    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-podinfo\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.664785 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.664605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-certs\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.664785 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.664650    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-envoy\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.765970 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.765935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-socket\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.765970 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.765972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-data\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766256 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.765996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-credential-socket\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766256 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.766022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-podinfo\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766256 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.766048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-certs\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766256 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.766082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-envoy\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766256 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.766149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bbfd53-33e2-4320-a5ed-2dac82182143-istiod-ca-cert\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766256 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.766209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-token\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766256 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.766254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zwbk\" (UniqueName: \"kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-kube-api-access-4zwbk\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766631 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.766446    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-data\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766631 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.766518    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-certs\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766631 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.766551    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-socket\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766790 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.766736    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-credential-socket\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.766945 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.766925    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bbfd53-33e2-4320-a5ed-2dac82182143-istiod-ca-cert\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.768512 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.768484    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-envoy\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.768626 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.768569    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-podinfo\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.774862 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.774819    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zwbk\" (UniqueName: \"kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-kube-api-access-4zwbk\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.775915 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.775895    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-token\") pod \"e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") " pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:28.918222 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:28.918158    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:29.059996 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:29.059961    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"]
Jul 06 20:27:29.062220 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:27:29.062195    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6bbfd53_33e2_4320_a5ed_2dac82182143.slice/crio-2e225524244658b65b1b29e4a0c1b21a5afb3f0b11dcea53d73bcf6c7283520c WatchSource:0}: Error finding container 2e225524244658b65b1b29e4a0c1b21a5afb3f0b11dcea53d73bcf6c7283520c: Status 404 returned error can't find the container with id 2e225524244658b65b1b29e4a0c1b21a5afb3f0b11dcea53d73bcf6c7283520c
Jul 06 20:27:29.064408 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:29.064379    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:29.064477 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:29.064447    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:29.064516 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:29.064476    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:29.539050 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:29.539013    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" event={"ID":"f6bbfd53-33e2-4320-a5ed-2dac82182143","Type":"ContainerStarted","Data":"905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803"}
Jul 06 20:27:29.539247 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:29.539068    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" event={"ID":"f6bbfd53-33e2-4320-a5ed-2dac82182143","Type":"ContainerStarted","Data":"2e225524244658b65b1b29e4a0c1b21a5afb3f0b11dcea53d73bcf6c7283520c"}
Jul 06 20:27:29.563901 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:29.563839    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" podStartSLOduration=1.5638247600000001 podStartE2EDuration="1.56382476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:27:29.562338932 +0000 UTC m=+2229.873794154" watchObservedRunningTime="2026-07-06 20:27:29.56382476 +0000 UTC m=+2229.875280034"
Jul 06 20:27:29.918450 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:29.918412    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:30.923672 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:30.923643    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:31.548820 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.548786    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:31.549897 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.549873    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:31.829886 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.829787    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"]
Jul 06 20:27:31.830477 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.830454    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e647436-ac15-4e56-a71e-31b822121170" containerName="istio-proxy"
Jul 06 20:27:31.834356 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.834331    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:31.836994 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.836854    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-45b15a-serving-cert\""
Jul 06 20:27:31.837155 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.837124    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mthr8\""
Jul 06 20:27:31.858382 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.858357    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"]
Jul 06 20:27:31.893084 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.893052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a68f1e7-1ffa-4678-bc07-17f730397680-maas-api-tls\") pod \"maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") " pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:31.893307 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.893112    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a68f1e7-1ffa-4678-bc07-17f730397680-openshift-service-ca\") pod \"maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") " pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:31.893307 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.893227    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pd9c\" (UniqueName: \"kubernetes.io/projected/5a68f1e7-1ffa-4678-bc07-17f730397680-kube-api-access-9pd9c\") pod \"maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") " pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:31.994618 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.994585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a68f1e7-1ffa-4678-bc07-17f730397680-maas-api-tls\") pod \"maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") " pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:31.995024 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.994649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a68f1e7-1ffa-4678-bc07-17f730397680-openshift-service-ca\") pod \"maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") " pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:31.995024 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.994679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pd9c\" (UniqueName: \"kubernetes.io/projected/5a68f1e7-1ffa-4678-bc07-17f730397680-kube-api-access-9pd9c\") pod \"maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") " pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:31.995024 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:27:31.994752    2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-45b15a-serving-cert: secret "maas-api-e2e-shared-b-45b15a-serving-cert" not found
Jul 06 20:27:31.995024 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:27:31.994822    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a68f1e7-1ffa-4678-bc07-17f730397680-maas-api-tls podName:5a68f1e7-1ffa-4678-bc07-17f730397680 nodeName:}" failed. No retries permitted until 2026-07-06 20:27:32.494803028 +0000 UTC m=+2232.806258227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a68f1e7-1ffa-4678-bc07-17f730397680-maas-api-tls") pod "maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c" (UID: "5a68f1e7-1ffa-4678-bc07-17f730397680") : secret "maas-api-e2e-shared-b-45b15a-serving-cert" not found
Jul 06 20:27:31.995333 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:31.995314    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a68f1e7-1ffa-4678-bc07-17f730397680-openshift-service-ca\") pod \"maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") " pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:32.035200 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:32.032940    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pd9c\" (UniqueName: \"kubernetes.io/projected/5a68f1e7-1ffa-4678-bc07-17f730397680-kube-api-access-9pd9c\") pod \"maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") " pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:32.499509 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:32.499473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a68f1e7-1ffa-4678-bc07-17f730397680-maas-api-tls\") pod \"maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") " pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:32.501905 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:32.501883    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a68f1e7-1ffa-4678-bc07-17f730397680-maas-api-tls\") pod \"maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") " pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:32.749156 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:32.749120    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:32.927668 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:32.924373    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"]
Jul 06 20:27:33.559101 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:33.559012    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c" event={"ID":"5a68f1e7-1ffa-4678-bc07-17f730397680","Type":"ContainerStarted","Data":"ecb9bd26ec0f011d5d28b3c83de7d74feaf90300c4eb0965f42fa8f91fa63c9f"}
Jul 06 20:27:33.559101 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:33.559073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c" event={"ID":"5a68f1e7-1ffa-4678-bc07-17f730397680","Type":"ContainerStarted","Data":"d233d334343733948470126791c62226a651bb819b485763c62c992784578073"}
Jul 06 20:27:33.559539 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:33.559200    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:33.584458 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:33.584412    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c" podStartSLOduration=2.22869607 podStartE2EDuration="2.584398509s" totalImagesPullingTime="355.702439ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:27:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:27:33.584217369 +0000 UTC m=+2233.895672590" watchObservedRunningTime="2026-07-06 20:27:33.584398509 +0000 UTC m=+2233.895853767"
Jul 06 20:27:38.270809 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:38.270560    2577 patch_prober.go:28] interesting pod/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": read tcp 10.134.0.2:57222->10.134.0.42:15021: read: connection reset by peer" start-of-body=
Jul 06 20:27:38.270809 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:38.270625    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": read tcp 10.134.0.2:57222->10.134.0.42:15021: read: connection reset by peer"
Jul 06 20:27:38.581373 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:38.581270    2577 generic.go:356] "Generic (PLEG): container finished" podID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerID="c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856" exitCode=0
Jul 06 20:27:38.581373 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:38.581346    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" event={"ID":"a01634cc-f054-4811-b4a8-cc1b8bdd2120","Type":"ContainerDied","Data":"c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856"}
Jul 06 20:27:38.581797 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:38.581779    2577 scope.go:117] "RemoveContainer" containerID="c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856"
Jul 06 20:27:38.582827 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:38.582801    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:38.582900 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:38.582873    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:38.582937 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:38.582912    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:39.568768 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:39.568740    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:27:39.587534 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:39.587504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" event={"ID":"a01634cc-f054-4811-b4a8-cc1b8bdd2120","Type":"ContainerStarted","Data":"956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa"}
Jul 06 20:27:40.244526 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:40.244489    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:40.244933 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:40.244845    2577 patch_prober.go:28] interesting pod/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 06 20:27:40.245127 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:40.245103    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 06 20:27:41.244026 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:41.243561    2577 patch_prober.go:28] interesting pod/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 06 20:27:41.244026 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:41.243632    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 06 20:27:42.243558 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:42.243517    2577 patch_prober.go:28] interesting pod/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 06 20:27:42.243862 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:42.243829    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 06 20:27:43.243683 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:43.243639    2577 patch_prober.go:28] interesting pod/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 06 20:27:43.244207 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:43.243714    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 06 20:27:44.247300 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:44.247263    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:44.247758 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:44.247644    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:44.248355 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:44.248336    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:27:46.550692 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:46.550659    2577 patch_prober.go:28] interesting pod/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 06 20:27:46.551126 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:46.550710    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 06 20:27:46.622193 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:46.622141    2577 generic.go:356] "Generic (PLEG): container finished" podID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerID="905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803" exitCode=0
Jul 06 20:27:46.622357 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:46.622208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" event={"ID":"f6bbfd53-33e2-4320-a5ed-2dac82182143","Type":"ContainerDied","Data":"905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803"}
Jul 06 20:27:46.622640 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:46.622627    2577 scope.go:117] "RemoveContainer" containerID="905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803"
Jul 06 20:27:46.623634 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:46.623611    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:46.623703 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:46.623694    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:46.623745 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:46.623725    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 06 20:27:47.632276 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:47.632231    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" event={"ID":"f6bbfd53-33e2-4320-a5ed-2dac82182143","Type":"ContainerStarted","Data":"7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e"}
Jul 06 20:27:47.920104 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:47.919549    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:47.920104 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:47.919712    2577 patch_prober.go:28] interesting pod/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 06 20:27:47.920104 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:47.919784    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 06 20:27:48.919022 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:48.918968    2577 patch_prober.go:28] interesting pod/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 06 20:27:48.919567 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:48.919045    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 06 20:27:49.919395 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:49.918923    2577 patch_prober.go:28] interesting pod/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 06 20:27:49.919395 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:49.918989    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 06 20:27:50.919062 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:50.919010    2577 patch_prober.go:28] interesting pod/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 06 20:27:50.919292 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:50.919094    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 06 20:27:51.922611 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:51.922584    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:51.923026 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:51.922853    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:27:51.923543 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:27:51.923523    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:31:20.433549 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:31:20.433519    2577 scope.go:117] "RemoveContainer" containerID="038eb0fe44b2605f0aeb36bff6096ed9c7522676b068176ab06f0cb28b8775f1"
Jul 06 20:33:43.619540 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:43.619453    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"]
Jul 06 20:33:43.620217 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:43.619742    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy" containerID="cri-o://956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa" gracePeriod=30
Jul 06 20:33:44.250083 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:44.250043    2577 patch_prober.go:28] interesting pod/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 20:33:44.250275 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:44.250114    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 20:33:44.786304 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:44.786270    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"]
Jul 06 20:33:44.786716 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:44.786517    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c" podUID="5a68f1e7-1ffa-4678-bc07-17f730397680" containerName="maas-api" containerID="cri-o://ecb9bd26ec0f011d5d28b3c83de7d74feaf90300c4eb0965f42fa8f91fa63c9f" gracePeriod=30
Jul 06 20:33:45.226781 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.226747    2577 generic.go:356] "Generic (PLEG): container finished" podID="5a68f1e7-1ffa-4678-bc07-17f730397680" containerID="ecb9bd26ec0f011d5d28b3c83de7d74feaf90300c4eb0965f42fa8f91fa63c9f" exitCode=0
Jul 06 20:33:45.226926 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.226827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c" event={"ID":"5a68f1e7-1ffa-4678-bc07-17f730397680","Type":"ContainerDied","Data":"ecb9bd26ec0f011d5d28b3c83de7d74feaf90300c4eb0965f42fa8f91fa63c9f"}
Jul 06 20:33:45.226926 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.226872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c" event={"ID":"5a68f1e7-1ffa-4678-bc07-17f730397680","Type":"ContainerDied","Data":"d233d334343733948470126791c62226a651bb819b485763c62c992784578073"}
Jul 06 20:33:45.226926 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.226889    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d233d334343733948470126791c62226a651bb819b485763c62c992784578073"
Jul 06 20:33:45.231505 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.231486    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:33:45.251541 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.251508    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a68f1e7-1ffa-4678-bc07-17f730397680-maas-api-tls\") pod \"5a68f1e7-1ffa-4678-bc07-17f730397680\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") "
Jul 06 20:33:45.251686 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.251554    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pd9c\" (UniqueName: \"kubernetes.io/projected/5a68f1e7-1ffa-4678-bc07-17f730397680-kube-api-access-9pd9c\") pod \"5a68f1e7-1ffa-4678-bc07-17f730397680\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") "
Jul 06 20:33:45.251686 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.251666    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a68f1e7-1ffa-4678-bc07-17f730397680-openshift-service-ca\") pod \"5a68f1e7-1ffa-4678-bc07-17f730397680\" (UID: \"5a68f1e7-1ffa-4678-bc07-17f730397680\") "
Jul 06 20:33:45.252278 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.252226    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a68f1e7-1ffa-4678-bc07-17f730397680-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a68f1e7-1ffa-4678-bc07-17f730397680" (UID: "5a68f1e7-1ffa-4678-bc07-17f730397680"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:33:45.254021 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.253984    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a68f1e7-1ffa-4678-bc07-17f730397680-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a68f1e7-1ffa-4678-bc07-17f730397680" (UID: "5a68f1e7-1ffa-4678-bc07-17f730397680"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:33:45.254272 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.254246    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a68f1e7-1ffa-4678-bc07-17f730397680-kube-api-access-9pd9c" (OuterVolumeSpecName: "kube-api-access-9pd9c") pod "5a68f1e7-1ffa-4678-bc07-17f730397680" (UID: "5a68f1e7-1ffa-4678-bc07-17f730397680"). InnerVolumeSpecName "kube-api-access-9pd9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:33:45.353002 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.352920    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a68f1e7-1ffa-4678-bc07-17f730397680-openshift-service-ca\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:45.353002 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.352949    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a68f1e7-1ffa-4678-bc07-17f730397680-maas-api-tls\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:45.353002 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:45.352964    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pd9c\" (UniqueName: \"kubernetes.io/projected/5a68f1e7-1ffa-4678-bc07-17f730397680-kube-api-access-9pd9c\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:46.231184 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:46.231139    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"
Jul 06 20:33:46.251348 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:46.251319    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"]
Jul 06 20:33:46.256770 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:46.256748    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-45b15a-6d9b87c4cf-9vw2c"]
Jul 06 20:33:46.337290 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:46.337260    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a68f1e7-1ffa-4678-bc07-17f730397680" path="/var/lib/kubelet/pods/5a68f1e7-1ffa-4678-bc07-17f730397680/volumes"
Jul 06 20:33:48.868435 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.868409    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:33:48.882868 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.882837    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-credential-socket\") pod \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") "
Jul 06 20:33:48.882993 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.882887    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-socket\") pod \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") "
Jul 06 20:33:48.882993 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.882951    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-data\") pod \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") "
Jul 06 20:33:48.882993 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.882974    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istiod-ca-cert\") pod \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") "
Jul 06 20:33:48.883120 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.882998    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-envoy\") pod \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") "
Jul 06 20:33:48.883120 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.883022    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-certs\") pod \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") "
Jul 06 20:33:48.883120 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.883046    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-token\") pod \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") "
Jul 06 20:33:48.883120 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.883082    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mshr\" (UniqueName: \"kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-kube-api-access-5mshr\") pod \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") "
Jul 06 20:33:48.883120 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.883114    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-podinfo\") pod \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\" (UID: \"a01634cc-f054-4811-b4a8-cc1b8bdd2120\") "
Jul 06 20:33:48.883400 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.883136    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a01634cc-f054-4811-b4a8-cc1b8bdd2120" (UID: "a01634cc-f054-4811-b4a8-cc1b8bdd2120"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:33:48.883400 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.883245    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a01634cc-f054-4811-b4a8-cc1b8bdd2120" (UID: "a01634cc-f054-4811-b4a8-cc1b8bdd2120"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:33:48.883400 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.883326    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a01634cc-f054-4811-b4a8-cc1b8bdd2120" (UID: "a01634cc-f054-4811-b4a8-cc1b8bdd2120"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:33:48.883590 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.883422    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-certs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:48.883590 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.883424    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a01634cc-f054-4811-b4a8-cc1b8bdd2120" (UID: "a01634cc-f054-4811-b4a8-cc1b8bdd2120"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:33:48.883590 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.883437    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-credential-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:48.883590 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.883451    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-workload-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:48.884240 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.884211    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-data" (OuterVolumeSpecName: "istio-data") pod "a01634cc-f054-4811-b4a8-cc1b8bdd2120" (UID: "a01634cc-f054-4811-b4a8-cc1b8bdd2120"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:33:48.885395 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.885373    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a01634cc-f054-4811-b4a8-cc1b8bdd2120" (UID: "a01634cc-f054-4811-b4a8-cc1b8bdd2120"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 20:33:48.885485 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.885462    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a01634cc-f054-4811-b4a8-cc1b8bdd2120" (UID: "a01634cc-f054-4811-b4a8-cc1b8bdd2120"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:33:48.885685 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.885666    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-kube-api-access-5mshr" (OuterVolumeSpecName: "kube-api-access-5mshr") pod "a01634cc-f054-4811-b4a8-cc1b8bdd2120" (UID: "a01634cc-f054-4811-b4a8-cc1b8bdd2120"). InnerVolumeSpecName "kube-api-access-5mshr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:33:48.885724 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.885694    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-token" (OuterVolumeSpecName: "istio-token") pod "a01634cc-f054-4811-b4a8-cc1b8bdd2120" (UID: "a01634cc-f054-4811-b4a8-cc1b8bdd2120"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:33:48.983875 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.983831    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-token\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:48.983875 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.983868    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mshr\" (UniqueName: \"kubernetes.io/projected/a01634cc-f054-4811-b4a8-cc1b8bdd2120-kube-api-access-5mshr\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:48.983875 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.983882    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-podinfo\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:48.984121 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.983893    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-data\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:48.984121 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.983904    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istiod-ca-cert\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:48.984121 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:48.983913    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a01634cc-f054-4811-b4a8-cc1b8bdd2120-istio-envoy\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:49.245312 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.245222    2577 generic.go:356] "Generic (PLEG): container finished" podID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerID="956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa" exitCode=0
Jul 06 20:33:49.245312 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.245275    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"
Jul 06 20:33:49.245312 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.245293    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" event={"ID":"a01634cc-f054-4811-b4a8-cc1b8bdd2120","Type":"ContainerDied","Data":"956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa"}
Jul 06 20:33:49.245568 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.245337    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5" event={"ID":"a01634cc-f054-4811-b4a8-cc1b8bdd2120","Type":"ContainerDied","Data":"4663831e3ad084368ed3cee7d5ad563167d09244fb0ab68de6aca99d83924983"}
Jul 06 20:33:49.245568 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.245361    2577 scope.go:117] "RemoveContainer" containerID="956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa"
Jul 06 20:33:49.254894 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.254873    2577 scope.go:117] "RemoveContainer" containerID="c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856"
Jul 06 20:33:49.262756 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.262733    2577 scope.go:117] "RemoveContainer" containerID="956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa"
Jul 06 20:33:49.263049 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:33:49.263025    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa\": container with ID starting with 956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa not found: ID does not exist" containerID="956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa"
Jul 06 20:33:49.263122 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.263059    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa"} err="failed to get container status \"956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa\": rpc error: code = NotFound desc = could not find container \"956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa\": container with ID starting with 956bafdb36dff8c766577230d56f4f63e41fe3d6b58f6898d0e092b3cc1595fa not found: ID does not exist"
Jul 06 20:33:49.263122 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.263079    2577 scope.go:117] "RemoveContainer" containerID="c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856"
Jul 06 20:33:49.263335 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:33:49.263316    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856\": container with ID starting with c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856 not found: ID does not exist" containerID="c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856"
Jul 06 20:33:49.263384 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.263340    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856"} err="failed to get container status \"c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856\": rpc error: code = NotFound desc = could not find container \"c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856\": container with ID starting with c5d97c04e702a41489050a3c9c1b581a1803239b7cf2688f4165c224a525a856 not found: ID does not exist"
Jul 06 20:33:49.266900 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.266876    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"]
Jul 06 20:33:49.268829 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:49.268807    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-482add-openshift-default-5dff479d59-pbdz5"]
Jul 06 20:33:50.337625 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:50.337577    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" path="/var/lib/kubelet/pods/a01634cc-f054-4811-b4a8-cc1b8bdd2120/volumes"
Jul 06 20:33:50.357752 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:50.357723    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"]
Jul 06 20:33:50.357967 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:50.357944    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy" containerID="cri-o://7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e" gracePeriod=30
Jul 06 20:33:51.925556 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:51.925525    2577 patch_prober.go:28] interesting pod/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 20:33:51.925992 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:51.925601    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 20:33:55.611187 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.611148    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:33:55.647801 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.647735    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-socket\") pod \"f6bbfd53-33e2-4320-a5ed-2dac82182143\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") "
Jul 06 20:33:55.647801 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.647771    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bbfd53-33e2-4320-a5ed-2dac82182143-istiod-ca-cert\") pod \"f6bbfd53-33e2-4320-a5ed-2dac82182143\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") "
Jul 06 20:33:55.647801 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.647795    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-podinfo\") pod \"f6bbfd53-33e2-4320-a5ed-2dac82182143\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") "
Jul 06 20:33:55.648067 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.647835    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-certs\") pod \"f6bbfd53-33e2-4320-a5ed-2dac82182143\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") "
Jul 06 20:33:55.648067 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.647894    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zwbk\" (UniqueName: \"kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-kube-api-access-4zwbk\") pod \"f6bbfd53-33e2-4320-a5ed-2dac82182143\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") "
Jul 06 20:33:55.648067 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.647919    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-data\") pod \"f6bbfd53-33e2-4320-a5ed-2dac82182143\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") "
Jul 06 20:33:55.648067 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.647955    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-envoy\") pod \"f6bbfd53-33e2-4320-a5ed-2dac82182143\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") "
Jul 06 20:33:55.648067 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.647980    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-token\") pod \"f6bbfd53-33e2-4320-a5ed-2dac82182143\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") "
Jul 06 20:33:55.648067 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.648007    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-credential-socket\") pod \"f6bbfd53-33e2-4320-a5ed-2dac82182143\" (UID: \"f6bbfd53-33e2-4320-a5ed-2dac82182143\") "
Jul 06 20:33:55.648408 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.648059    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f6bbfd53-33e2-4320-a5ed-2dac82182143" (UID: "f6bbfd53-33e2-4320-a5ed-2dac82182143"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:33:55.648408 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.648269    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6bbfd53-33e2-4320-a5ed-2dac82182143-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f6bbfd53-33e2-4320-a5ed-2dac82182143" (UID: "f6bbfd53-33e2-4320-a5ed-2dac82182143"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:33:55.648408 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.648357    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f6bbfd53-33e2-4320-a5ed-2dac82182143" (UID: "f6bbfd53-33e2-4320-a5ed-2dac82182143"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:33:55.648667 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.648599    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f6bbfd53-33e2-4320-a5ed-2dac82182143" (UID: "f6bbfd53-33e2-4320-a5ed-2dac82182143"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:33:55.648759 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.648700    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:55.648759 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.648715    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bbfd53-33e2-4320-a5ed-2dac82182143-istiod-ca-cert\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:55.648759 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.648727    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-workload-certs\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:55.648759 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.648740    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-credential-socket\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:55.649271 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.649245    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-data" (OuterVolumeSpecName: "istio-data") pod "f6bbfd53-33e2-4320-a5ed-2dac82182143" (UID: "f6bbfd53-33e2-4320-a5ed-2dac82182143"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:33:55.650263 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.650239    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f6bbfd53-33e2-4320-a5ed-2dac82182143" (UID: "f6bbfd53-33e2-4320-a5ed-2dac82182143"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 20:33:55.650349 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.650302    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6bbfd53-33e2-4320-a5ed-2dac82182143" (UID: "f6bbfd53-33e2-4320-a5ed-2dac82182143"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:33:55.650529 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.650510    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-kube-api-access-4zwbk" (OuterVolumeSpecName: "kube-api-access-4zwbk") pod "f6bbfd53-33e2-4320-a5ed-2dac82182143" (UID: "f6bbfd53-33e2-4320-a5ed-2dac82182143"). InnerVolumeSpecName "kube-api-access-4zwbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:33:55.650598 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.650558    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f6bbfd53-33e2-4320-a5ed-2dac82182143" (UID: "f6bbfd53-33e2-4320-a5ed-2dac82182143"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:33:55.693123 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.693093    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qzc5s_2372363d-da51-48d9-a704-014f6c2684d3/manager/0.log"
Jul 06 20:33:55.750194 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.750143    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zwbk\" (UniqueName: \"kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-kube-api-access-4zwbk\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:55.750194 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.750169    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-data\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:55.750194 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.750200    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-envoy\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:55.750439 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.750212    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-token\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:55.750439 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.750224    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bbfd53-33e2-4320-a5ed-2dac82182143-istio-podinfo\") on node \"ip-10-0-139-90.ec2.internal\" DevicePath \"\""
Jul 06 20:33:55.821553 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:55.821510    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bgtgn_d724be70-74c7-4ecb-9da7-f3a5ca4c19ab/manager/0.log"
Jul 06 20:33:56.277919 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.277887    2577 generic.go:356] "Generic (PLEG): container finished" podID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerID="7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e" exitCode=0
Jul 06 20:33:56.277919 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.277922    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" event={"ID":"f6bbfd53-33e2-4320-a5ed-2dac82182143","Type":"ContainerDied","Data":"7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e"}
Jul 06 20:33:56.278203 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.277939    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"
Jul 06 20:33:56.278203 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.277963    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g" event={"ID":"f6bbfd53-33e2-4320-a5ed-2dac82182143","Type":"ContainerDied","Data":"2e225524244658b65b1b29e4a0c1b21a5afb3f0b11dcea53d73bcf6c7283520c"}
Jul 06 20:33:56.278203 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.277983    2577 scope.go:117] "RemoveContainer" containerID="7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e"
Jul 06 20:33:56.286864 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.286846    2577 scope.go:117] "RemoveContainer" containerID="905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803"
Jul 06 20:33:56.294915 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.294896    2577 scope.go:117] "RemoveContainer" containerID="7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e"
Jul 06 20:33:56.295212 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:33:56.295192    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e\": container with ID starting with 7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e not found: ID does not exist" containerID="7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e"
Jul 06 20:33:56.295304 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.295221    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e"} err="failed to get container status \"7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e\": rpc error: code = NotFound desc = could not find container \"7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e\": container with ID starting with 7bb0b07882c701bd1126cc48b82eb242426e4d70bcc0f4761ed3e32ff56a3f4e not found: ID does not exist"
Jul 06 20:33:56.295304 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.295245    2577 scope.go:117] "RemoveContainer" containerID="905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803"
Jul 06 20:33:56.295654 ip-10-0-139-90 kubenswrapper[2577]: E0706 20:33:56.295610    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803\": container with ID starting with 905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803 not found: ID does not exist" containerID="905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803"
Jul 06 20:33:56.295654 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.295644    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803"} err="failed to get container status \"905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803\": rpc error: code = NotFound desc = could not find container \"905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803\": container with ID starting with 905c659b103f6588875b78e4c35801e16a07cc47c02451dda9524e9b7d570803 not found: ID does not exist"
Jul 06 20:33:56.299002 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.298980    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"]
Jul 06 20:33:56.304957 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.304937    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-45b15a-openshift-default-6fddc9f9-2m24g"]
Jul 06 20:33:56.337375 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.337349    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" path="/var/lib/kubelet/pods/f6bbfd53-33e2-4320-a5ed-2dac82182143/volumes"
Jul 06 20:33:56.809574 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.809545    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87fbc8964-wjg49_0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de/copy-manifests/0.log"
Jul 06 20:33:56.816953 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:56.816926    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87fbc8964-wjg49_0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de/manager/0.log"
Jul 06 20:33:58.948782 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:58.948757    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-l557g_f2360391-eee4-4c7a-baa7-f7664d420110/manager/0.log"
Jul 06 20:33:59.195756 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:33:59.195728    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lztrv_ff0ce936-bf25-4608-bdae-ee23d8bd9b2a/manager/0.log"
Jul 06 20:34:00.166669 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:00.166640    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-xwd5l_b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5/payload-pre-processing/0.log"
Jul 06 20:34:01.022909 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:01.022871    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-98crx_eb318378-0c93-4df6-8254-1e80b227445b/main/0.log"
Jul 06 20:34:01.153447 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:01.153415    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwvj6d_302d0400-1481-46c3-8ce6-b0ea575ba77d/main/0.log"
Jul 06 20:34:08.812096 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:08.812058    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8x5db_6feda960-34dc-47d7-bd85-4cbbc95bf5ba/global-pull-secret-syncer/0.log"
Jul 06 20:34:09.016419 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:09.016389    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x9qff_f4b68573-84ea-4f3d-980c-882767d83501/konnectivity-agent/0.log"
Jul 06 20:34:09.129607 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:09.129575    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-90.ec2.internal_36e6979a7a16f66ce876f6ff5066432c/haproxy/0.log"
Jul 06 20:34:15.079466 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:15.079428    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-l557g_f2360391-eee4-4c7a-baa7-f7664d420110/manager/0.log"
Jul 06 20:34:15.175783 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:15.175739    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lztrv_ff0ce936-bf25-4608-bdae-ee23d8bd9b2a/manager/0.log"
Jul 06 20:34:17.264140 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:17.264109    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h7nr5_2407df6e-673c-4112-a05d-2d677966703b/node-exporter/0.log"
Jul 06 20:34:17.286000 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:17.285971    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h7nr5_2407df6e-673c-4112-a05d-2d677966703b/kube-rbac-proxy/0.log"
Jul 06 20:34:17.317033 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:17.317008    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h7nr5_2407df6e-673c-4112-a05d-2d677966703b/init-textfile/0.log"
Jul 06 20:34:17.684681 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:17.684654    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57b648be-e24d-47cd-b26e-9c6c65040474/prometheus/0.log"
Jul 06 20:34:17.708872 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:17.708848    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57b648be-e24d-47cd-b26e-9c6c65040474/config-reloader/0.log"
Jul 06 20:34:17.733320 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:17.733294    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57b648be-e24d-47cd-b26e-9c6c65040474/thanos-sidecar/0.log"
Jul 06 20:34:17.766224 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:17.766191    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57b648be-e24d-47cd-b26e-9c6c65040474/kube-rbac-proxy-web/0.log"
Jul 06 20:34:17.802037 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:17.802005    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57b648be-e24d-47cd-b26e-9c6c65040474/kube-rbac-proxy/0.log"
Jul 06 20:34:17.818189 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:17.818147    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57b648be-e24d-47cd-b26e-9c6c65040474/kube-rbac-proxy-thanos/0.log"
Jul 06 20:34:17.840277 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:17.840250    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57b648be-e24d-47cd-b26e-9c6c65040474/init-config-reloader/0.log"
Jul 06 20:34:19.126924 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.126894    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"]
Jul 06 20:34:19.127316 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127216    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy"
Jul 06 20:34:19.127316 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127227    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy"
Jul 06 20:34:19.127316 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127235    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy"
Jul 06 20:34:19.127316 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127241    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy"
Jul 06 20:34:19.127316 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127253    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy"
Jul 06 20:34:19.127316 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127259    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy"
Jul 06 20:34:19.127316 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127271    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy"
Jul 06 20:34:19.127316 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127277    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy"
Jul 06 20:34:19.127316 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127283    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a68f1e7-1ffa-4678-bc07-17f730397680" containerName="maas-api"
Jul 06 20:34:19.127316 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127289    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a68f1e7-1ffa-4678-bc07-17f730397680" containerName="maas-api"
Jul 06 20:34:19.127626 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127334    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a68f1e7-1ffa-4678-bc07-17f730397680" containerName="maas-api"
Jul 06 20:34:19.127626 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127343    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy"
Jul 06 20:34:19.127626 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127349    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bbfd53-33e2-4320-a5ed-2dac82182143" containerName="istio-proxy"
Jul 06 20:34:19.127626 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127355    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy"
Jul 06 20:34:19.127626 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.127359    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a01634cc-f054-4811-b4a8-cc1b8bdd2120" containerName="istio-proxy"
Jul 06 20:34:19.130361 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.130346    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.133105 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.133082    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-282xb\"/\"kube-root-ca.crt\""
Jul 06 20:34:19.133315 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.133118    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-282xb\"/\"openshift-service-ca.crt\""
Jul 06 20:34:19.133315 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.133253    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-282xb\"/\"default-dockercfg-vsdcj\""
Jul 06 20:34:19.144430 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.144407    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"]
Jul 06 20:34:19.251111 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.251074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-sys\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.251111 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.251110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-proc\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.251362 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.251224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-podres\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.251362 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.251267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-lib-modules\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.251362 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.251297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmvvj\" (UniqueName: \"kubernetes.io/projected/6a3974df-90de-4c3c-a47b-668b07775c48-kube-api-access-pmvvj\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.351683 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.351649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmvvj\" (UniqueName: \"kubernetes.io/projected/6a3974df-90de-4c3c-a47b-668b07775c48-kube-api-access-pmvvj\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.351849 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.351719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-sys\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.351849 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.351744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-proc\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.351849 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.351796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-podres\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.351849 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.351833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-lib-modules\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.352021 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.351853    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-sys\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.352021 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.351868    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-proc\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.352021 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.351947    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-podres\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.352021 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.351966    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a3974df-90de-4c3c-a47b-668b07775c48-lib-modules\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.359672 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.359647    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmvvj\" (UniqueName: \"kubernetes.io/projected/6a3974df-90de-4c3c-a47b-668b07775c48-kube-api-access-pmvvj\") pod \"perf-node-gather-daemonset-2tqxk\" (UID: \"6a3974df-90de-4c3c-a47b-668b07775c48\") " pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.440133 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.440043    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:19.569725 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:19.569693    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"]
Jul 06 20:34:19.573730 ip-10-0-139-90 kubenswrapper[2577]: W0706 20:34:19.573698    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6a3974df_90de_4c3c_a47b_668b07775c48.slice/crio-5ca763c5c014fa21659f25491a89b05f3c8d01f400bd8ff520b8a1bd3d7b62b0 WatchSource:0}: Error finding container 5ca763c5c014fa21659f25491a89b05f3c8d01f400bd8ff520b8a1bd3d7b62b0: Status 404 returned error can't find the container with id 5ca763c5c014fa21659f25491a89b05f3c8d01f400bd8ff520b8a1bd3d7b62b0
Jul 06 20:34:20.380738 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:20.380701    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk" event={"ID":"6a3974df-90de-4c3c-a47b-668b07775c48","Type":"ContainerStarted","Data":"fd1af3b0739811b3de00504ce9f9558656c850196c5ba9601c9a81e88cbec4f1"}
Jul 06 20:34:20.381249 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:20.380754    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk" event={"ID":"6a3974df-90de-4c3c-a47b-668b07775c48","Type":"ContainerStarted","Data":"5ca763c5c014fa21659f25491a89b05f3c8d01f400bd8ff520b8a1bd3d7b62b0"}
Jul 06 20:34:20.381249 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:20.380790    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:20.398752 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:20.398691    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk" podStartSLOduration=1.3986278109999999 podStartE2EDuration="1.398627811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:34:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:34:20.398590434 +0000 UTC m=+2640.710045656" watchObservedRunningTime="2026-07-06 20:34:20.398627811 +0000 UTC m=+2640.710083029"
Jul 06 20:34:20.465470 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:20.465440    2577 scope.go:117] "RemoveContainer" containerID="ecb9bd26ec0f011d5d28b3c83de7d74feaf90300c4eb0965f42fa8f91fa63c9f"
Jul 06 20:34:22.049467 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:22.049442    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lqm67_35a35874-da0b-41fe-b1d6-25bd8082e927/dns/0.log"
Jul 06 20:34:22.071785 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:22.071753    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lqm67_35a35874-da0b-41fe-b1d6-25bd8082e927/kube-rbac-proxy/0.log"
Jul 06 20:34:22.180913 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:22.180886    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qzw6h_f2473042-4162-458d-8969-ae7e8668b0b3/dns-node-resolver/0.log"
Jul 06 20:34:22.751871 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:22.751841    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7887966bdc-54bwk_6418cafc-b74f-49ae-9c50-4537ac11d2ea/registry/0.log"
Jul 06 20:34:22.773010 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:22.772985    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-29qdv_509ae1fb-ff23-4a78-acb5-5f55e5900408/node-ca/0.log"
Jul 06 20:34:23.993090 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:23.993063    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-xwd5l_b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5/payload-pre-processing/0.log"
Jul 06 20:34:24.737132 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:24.737102    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rbkts_e0a787a4-c59e-48e3-b0ac-736de6af9b84/serve-healthcheck-canary/0.log"
Jul 06 20:34:25.271953 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:25.271919    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-79s77_b122086a-cb4b-4021-80ab-391c8097b4a1/kube-rbac-proxy/0.log"
Jul 06 20:34:25.294698 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:25.294672    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-79s77_b122086a-cb4b-4021-80ab-391c8097b4a1/exporter/0.log"
Jul 06 20:34:25.317003 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:25.316978    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-79s77_b122086a-cb4b-4021-80ab-391c8097b4a1/extractor/0.log"
Jul 06 20:34:26.393966 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:26.393943    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-282xb/perf-node-gather-daemonset-2tqxk"
Jul 06 20:34:27.672467 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:27.672439    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qzc5s_2372363d-da51-48d9-a704-014f6c2684d3/manager/0.log"
Jul 06 20:34:27.807783 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:27.807754    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bgtgn_d724be70-74c7-4ecb-9da7-f3a5ca4c19ab/manager/0.log"
Jul 06 20:34:28.297656 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:28.297619    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87fbc8964-wjg49_0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de/manager/0.log"
Jul 06 20:34:28.358962 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:28.358934    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87fbc8964-wjg49_0dda2bc0-02e7-4f52-9d9f-08d5ee0b12de/copy-manifests/0.log"
Jul 06 20:34:29.859410 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:29.859351    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65b8985978-jnprq_50da2f9f-5234-4a6d-a8bb-1466a6437377/manager/0.log"
Jul 06 20:34:29.919703 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:29.919673    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-xwd5l_b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5/payload-pre-processing/0.log"
Jul 06 20:34:31.470892 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:31.470824    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-xwd5l_b36cb15e-1d70-4b8a-8523-6fd9b5ab3dd5/payload-pre-processing/0.log"
Jul 06 20:34:36.272353 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:36.272325    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f42z_6207540f-5268-4df4-b0fa-94cd2e4e4a44/kube-multus-additional-cni-plugins/0.log"
Jul 06 20:34:36.297723 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:36.297692    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f42z_6207540f-5268-4df4-b0fa-94cd2e4e4a44/egress-router-binary-copy/0.log"
Jul 06 20:34:36.319724 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:36.319696    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f42z_6207540f-5268-4df4-b0fa-94cd2e4e4a44/cni-plugins/0.log"
Jul 06 20:34:36.341481 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:36.341458    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f42z_6207540f-5268-4df4-b0fa-94cd2e4e4a44/bond-cni-plugin/0.log"
Jul 06 20:34:36.363922 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:36.363900    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f42z_6207540f-5268-4df4-b0fa-94cd2e4e4a44/routeoverride-cni/0.log"
Jul 06 20:34:36.386135 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:36.386109    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f42z_6207540f-5268-4df4-b0fa-94cd2e4e4a44/whereabouts-cni-bincopy/0.log"
Jul 06 20:34:36.415041 ip-10-0-139-90 kubenswrapper[2577]: I0706 20:34:36.415020    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f42z_6207540f-5268-4df4-b0fa-94cd2e4e4a44/whereabouts-cni/0.log"