Jul 22 11:59:42.793906 ip-10-0-143-107 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 22 11:59:42.793920 ip-10-0-143-107 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 22 11:59:42.793930 ip-10-0-143-107 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 22 11:59:42.794349 ip-10-0-143-107 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 22 11:59:52.980151 ip-10-0-143-107 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 22 11:59:52.980166 ip-10-0-143-107 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b128806ca77447fa97972510bfa100d4 --
Jul 22 12:01:50.860657 ip-10-0-143-107 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 12:01:51.274181 ip-10-0-143-107 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:01:51.274181 ip-10-0-143-107 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 12:01:51.274181 ip-10-0-143-107 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:01:51.274181 ip-10-0-143-107 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 12:01:51.274181 ip-10-0-143-107 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:01:51.276076 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.275963    2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 12:01:51.281524 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281507    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:51.281524 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281524    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281529    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281532    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281537    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281542    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281545    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281548    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281551    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281554    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281557    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281560    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281562    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281565    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281567    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281571    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281573    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281576    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281582    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281585    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281587    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:51.281593 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281590    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281593    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281595    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281598    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281601    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281604    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281606    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281609    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281612    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281614    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281617    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281619    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281622    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281624    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281627    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281630    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281632    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281635    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281637    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281640    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:51.282055 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281642    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281645    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281647    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281649    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281652    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281655    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281657    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281660    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281663    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281667    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281671    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281674    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281677    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281680    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281683    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281686    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281688    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281691    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281693    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:51.282552 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281696    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281698    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281701    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281703    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281706    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281708    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281711    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281713    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281716    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281718    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281720    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281723    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281726    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281729    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281732    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281734    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281736    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281739    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281742    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:51.283011 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281745    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281749    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281752    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281755    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281758    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281761    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281764    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281766    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281769    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281772    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281774    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281776    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281779    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281781    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281784    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281786    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281789    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281791    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281794    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.281796    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:51.283470 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282499    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282507    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282510    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282513    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282516    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282519    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282521    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282524    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282527    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282529    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282532    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282534    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282537    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282540    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282542    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282544    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282547    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282550    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282553    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:51.283935 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282556    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282561    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282564    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282566    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282571    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282575    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282578    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282582    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282585    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282589    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282592    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282595    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282598    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282601    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282603    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282606    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282609    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282611    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282614    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:51.284413 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282617    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282619    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282622    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282624    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282627    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282629    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282632    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282634    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282637    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282640    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282643    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282646    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282648    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282652    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282655    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282657    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282660    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282663    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282665    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282668    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:51.284877 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282670    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282673    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282675    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282678    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282680    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282683    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282685    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282687    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282690    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282692    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282695    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282699    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282702    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282704    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282707    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282710    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282712    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282715    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282717    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282720    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:51.285342 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282724    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282726    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282729    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282732    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282734    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282737    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282741    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282743    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282746    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282748    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282751    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282753    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282756    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282758    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282762    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282765    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282768    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282771    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282773    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282776    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:51.285848 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.282778    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283157    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283162    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283165    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283168    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283170    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283175    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283184    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283187    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283190    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283192    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283195    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283198    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283200    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283203    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283205    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283209    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283212    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283219    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:51.286329 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283222    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283224    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283227    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283229    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283232    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283234    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283237    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283239    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283242    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283244    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283246    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283249    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283251    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283254    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283256    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283259    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283261    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283264    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283266    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283268    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:51.286800 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283271    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283273    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283275    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283278    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283280    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283283    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283286    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283288    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283291    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283294    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283297    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283300    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283303    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283306    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283308    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283311    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283313    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283315    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283318    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283321    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:51.287305 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283323    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283326    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283328    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283331    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283333    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283336    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283338    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283340    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283343    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283345    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283348    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283350    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283353    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283355    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283358    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283361    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283365    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283368    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283371    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:51.287778 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283388    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283391    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283394    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283396    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283399    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283401    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283405    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283408    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283410    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283413    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283415    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283418    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283420    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283423    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283426    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283428    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283431    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283434    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283436    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283439    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:51.288225 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283441    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.283444    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283512    2584 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283524    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283532    2584 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283538    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283548    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283552    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283557    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283560    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283563    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283567    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283571    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283574    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283577    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283580    2584 flags.go:64] FLAG: --cgroup-root=""
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283583    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283586    2584 flags.go:64] FLAG: --client-ca-file=""
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283589    2584 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283593    2584 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283597    2584 flags.go:64] FLAG: --cluster-domain=""
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283600    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283603    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283606    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 12:01:51.288751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283609    2584 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283613    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283616    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283619    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283622    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283625    2584 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283628    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283631    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283636    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283639    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283643    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283646    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283649    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283652    2584 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283656    2584 flags.go:64] FLAG: --enable-server="true"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283659    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283663    2584 flags.go:64] FLAG: --event-burst="100"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283666    2584 flags.go:64] FLAG: --event-qps="50"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283669    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283672    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283674    2584 flags.go:64] FLAG: --eviction-hard=""
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283678    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283681    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283684    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283687    2584 flags.go:64] FLAG: --eviction-soft=""
Jul 22 12:01:51.289320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283690    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283693    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283695    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283699    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283701    2584 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283704    2584 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283707    2584 flags.go:64] FLAG: --feature-gates=""
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283710    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283713    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283716    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283719    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283722    2584 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283725    2584 flags.go:64] FLAG: --help="false"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283728    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283731    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283735    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283738    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283741    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283745    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283747    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283750    2584 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283754    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283757    2584 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 12:01:51.289924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283760    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283763    2584 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283766    2584 flags.go:64] FLAG: --kube-reserved=""
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283768    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283771    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283774    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283777    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283780    2584 flags.go:64] FLAG: --lock-file=""
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283782    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283785    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283788    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283793    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283796    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283799    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283802    2584 flags.go:64] FLAG: --logging-format="text"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283805    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283808    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283811    2584 flags.go:64] FLAG: --manifest-url=""
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283813    2584 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283818    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283821    2584 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283825    2584 flags.go:64] FLAG: --max-pods="110"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283827    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283830    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283837    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 12:01:51.290523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283840    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283843    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283846    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283848    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283856    2584 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283860    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283863    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283866    2584 flags.go:64] FLAG: --pod-cidr=""
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283869    2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283874    2584 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283876    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283879    2584 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283882    2584 flags.go:64] FLAG: --port="10250"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283885    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283888    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0df4d257766ed21fe"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283891    2584 flags.go:64] FLAG: --qos-reserved=""
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283894    2584 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283897    2584 flags.go:64] FLAG: --register-node="true"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283900    2584 flags.go:64] FLAG: --register-with-taints=""
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283903    2584 flags.go:64] FLAG: --registry-burst="10"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283906    2584 flags.go:64] FLAG: --registry-qps="5"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283909    2584 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283912    2584 flags.go:64] FLAG: --reserved-memory=""
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283915    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283919    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 12:01:51.291117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283921    2584 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283924    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283927    2584 flags.go:64] FLAG: --runonce="false"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283930    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283933    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283936    2584 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283939    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283942    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283945    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283948    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283951    2584 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283954    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283957    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283961    2584 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283964    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283967    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283970    2584 flags.go:64] FLAG: --system-cgroups=""
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283973    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283978    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283981    2584 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283983    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283988    2584 flags.go:64] FLAG: --tls-min-version=""
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283990    2584 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283993    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283996    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 12:01:51.291729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.283998    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.284001    2584 flags.go:64] FLAG: --v="2"
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.284005    2584 flags.go:64] FLAG: --version="false"
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.284008    2584 flags.go:64] FLAG: --vmodule=""
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.284013    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.284016    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284132    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284136    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284139    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284142    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284145    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284148    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284151    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284155    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284159    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284162    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284165    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284167    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284170    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284172    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284177    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:51.292361 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284180    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284183    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284185    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284188    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284190    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284193    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284195    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284198    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284200    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284203    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284205    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284208    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284210    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284215    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284218    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284221    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284225    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284228    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284231    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284233    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:51.292875 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284236    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284239    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284242    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284244    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284247    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284249    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284252    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284255    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284257    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284260    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284262    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284267    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284269    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284272    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284275    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284277    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284280    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284283    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284285    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284288    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:51.293351 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284291    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284293    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284296    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284298    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284301    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284303    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284306    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284308    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284311    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284314    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284316    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284319    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284321    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284324    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284327    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284329    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284331    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284334    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284337    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:51.293912 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284339    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284341    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284344    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284347    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284351    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284353    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284356    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284360    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284363    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284366    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284369    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284372    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284388    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284391    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284394    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284397    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284400    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284403    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284406    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:51.294386 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284408    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284411    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284414    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284417    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284420    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.284423    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.285299    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.292690    2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.292704    2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.292722    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.292728    2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292804    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292809    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292813    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292816    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:51.294899 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292819    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292822    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292825    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292828    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292830    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292833    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292835    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292838    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292840    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292843    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292846    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292849    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292858    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292861    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292863    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292866    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292868    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292871    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292874    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292877    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:51.295266 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292879    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292882    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292885    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292888    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292891    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292893    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292897    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292899    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292902    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292904    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292907    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292910    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292913    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292915    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292918    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292921    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292923    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292926    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292928    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292931    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:51.295942 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292933    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292935    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292938    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292941    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292943    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292951    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292954    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292956    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292959    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292962    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292964    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292967    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292969    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292972    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292974    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292977    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292980    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292985    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292989    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292991    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:51.296843 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292994    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292997    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.292999    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293002    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293010    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293014    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293018    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293021    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293023    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293026    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293029    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293033    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293036    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293039    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293042    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293045    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293048    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293051    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293060    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:51.297371 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293063    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293066    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293068    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293071    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293073    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293076    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293078    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293081    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293083    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293086    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293090    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293092    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293095    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293097    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293100    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:51.297903 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293103    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.293108    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293254    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293259    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293262    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293264    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293267    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293270    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293273    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293275    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293278    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293281    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293284    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293286    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293289    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:51.298430 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293291    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293294    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293302    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293305    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293307    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293310    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293312    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293315    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293317    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293320    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293322    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293325    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293328    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293330    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293333    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293335    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293338    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293340    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293343    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:51.298941 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293345    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293348    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293350    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293353    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293355    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293358    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293361    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293363    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293366    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293369    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293371    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293390    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293394    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293396    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293399    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293402    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293412    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293416    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293419    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293422    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:51.299434 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293425    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293427    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293430    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293433    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293435    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293438    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293441    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293443    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293446    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293449    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293451    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293454    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293456    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293459    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293461    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293464    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293466    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293469    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293471    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293474    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:51.299966 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293476    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293479    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293481    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293484    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293486    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293489    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293491    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293495    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293499    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293508    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293512    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293514    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293517    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293520    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293523    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293525    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293528    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293531    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293533    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:51.300497 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293535    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293538    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293541    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293543    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293546    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293548    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293550    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:51.293553    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.293558    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.293650    2584 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.297723    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.299220    2584 server.go:1019] "Starting client certificate rotation"
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.299330    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 12:01:51.300984 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.299368    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 12:01:51.302063 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.302048    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 12:01:51.306878 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.306858    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 12:01:51.314709 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.314690    2584 log.go:25] "Validated CRI v1 runtime API"
Jul 22 12:01:51.318351 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.318332    2584 log.go:25] "Validated CRI v1 image API"
Jul 22 12:01:51.319228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.319212    2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 12:01:51.322161 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.322136    2584 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aafba068-2d31-4845-961c-68caf6b3bffe:/dev/nvme0n1p4 d14e53da-e00b-4e99-b7c1-5cbfce0a718a:/dev/nvme0n1p3]
Jul 22 12:01:51.322212 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.322162    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 12:01:51.328193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.328074    2584 manager.go:216] Machine: {Timestamp:2026-07-22 12:01:51.326606268 +0000 UTC m=+0.348627964 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100933 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a90fdd86c7490147e8f158f873756 SystemUUID:ec2a90fd-d86c-7490-147e-8f158f873756 BootID:b128806c-a774-47fa-9797-2510bfa100d4 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dd:78:bc:f6:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dd:78:bc:f6:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:ce:d1:51:76:e7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 12:01:51.328193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.328188    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 12:01:51.328303 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.328264    2584 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 12:01:51.331582 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.331555    2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 12:01:51.331726 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.331584    2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-107.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 12:01:51.331936 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.331919    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 12:01:51.332419 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.332407    2584 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 12:01:51.332469 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.332422    2584 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 12:01:51.332469 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.332429    2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 12:01:51.333121 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.333110    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 12:01:51.334475 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.334448    2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 12:01:51.337743 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.337731    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 12:01:51.337877 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.337866    2584 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 12:01:51.337911 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.337899    2584 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 12:01:51.337944 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.337913    2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 12:01:51.337944 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.337929    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 12:01:51.337944 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.337938    2584 kubelet.go:398] "Adding apiserver pod source"
Jul 22 12:01:51.338048 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.337946    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 12:01:51.338324 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.338311    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 12:01:51.338370 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.338332    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 12:01:51.340848 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.340816    2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 12:01:51.343435 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.343417    2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 12:01:51.343536 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.343470    2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 12:01:51.344252 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344236    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 12:01:51.344293 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344258    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 12:01:51.344293 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344265    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 12:01:51.344293 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344273    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 12:01:51.344293 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344286    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 12:01:51.344293 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344292    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 12:01:51.344435 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344393    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 12:01:51.344435 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344404    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 12:01:51.344435 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344412    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 12:01:51.344435 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344418    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 12:01:51.344435 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344428    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 12:01:51.344435 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.344436    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 12:01:51.345328 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.345317    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 12:01:51.345328 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.345328    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 12:01:51.348946 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.348908    2584 server.go:1268] "Started kubelet"
Jul 22 12:01:51.349033 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.348989    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 12:01:51.349128 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.349058    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 12:01:51.349231 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.349200    2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 12:01:51.349864 ip-10-0-143-107 systemd[1]: Started Kubernetes Kubelet.
Jul 22 12:01:51.350077 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.350059    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-107.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 12:01:51.350164 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.350106    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 12:01:51.350164 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.350143    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-107.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 12:01:51.350488 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.350398    2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 12:01:51.351522 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.351507    2584 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 12:01:51.355801 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.354776    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-107.ec2.internal.18c49af0cbe069bd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-107.ec2.internal,UID:ip-10-0-143-107.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-107.ec2.internal,},FirstTimestamp:2026-07-22 12:01:51.348869565 +0000 UTC m=+0.370891259,LastTimestamp:2026-07-22 12:01:51.348869565 +0000 UTC m=+0.370891259,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-107.ec2.internal,}"
Jul 22 12:01:51.357252 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.357235    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7dj24"
Jul 22 12:01:51.357684 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.357659    2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 12:01:51.357844 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.357827    2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 12:01:51.357844 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.357840    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 12:01:51.358020 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.357895    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:51.358591 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.358572    2584 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 12:01:51.358591 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.358593    2584 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 12:01:51.358743 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.358575    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 12:01:51.358743 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.358672    2584 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 12:01:51.358743 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.358681    2584 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 12:01:51.358884 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.358779    2584 factory.go:55] Registering systemd factory
Jul 22 12:01:51.358884 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.358797    2584 factory.go:223] Registration of the systemd container factory successfully
Jul 22 12:01:51.359053 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.359035    2584 factory.go:153] Registering CRI-O factory
Jul 22 12:01:51.359053 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.359051    2584 factory.go:223] Registration of the crio container factory successfully
Jul 22 12:01:51.359182 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.359092    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 12:01:51.359182 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.359111    2584 factory.go:103] Registering Raw factory
Jul 22 12:01:51.359182 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.359122    2584 manager.go:1196] Started watching for new ooms in manager
Jul 22 12:01:51.359586 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.359569    2584 manager.go:349] Starting recovery of all containers
Jul 22 12:01:51.363906 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.363872    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 22 12:01:51.363906 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.363877    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-107.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 22 12:01:51.367458 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.367420    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7dj24"
Jul 22 12:01:51.373448 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.373425    2584 manager.go:354] Recovery completed
Jul 22 12:01:51.378674 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.378660    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:51.380621 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.380603    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:51.380678 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.380635    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:51.380678 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.380646    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:51.381208 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.381190    2584 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 12:01:51.381208 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.381201    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:51.381208 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.381206    2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 12:01:51.381374 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.381222    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 12:01:51.381989 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.381964    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:51.382045 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.382000    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:51.382045 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.382013    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:51.382454 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.382439    2584 policy_none.go:49] "None policy: Start"
Jul 22 12:01:51.382454 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.382467    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 12:01:51.382454 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.382478    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 12:01:51.382742 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.382657    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-107.ec2.internal.18c49af0cdc4e9fc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-107.ec2.internal,UID:ip-10-0-143-107.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-107.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-107.ec2.internal,},FirstTimestamp:2026-07-22 12:01:51.38062182 +0000 UTC m=+0.402643514,LastTimestamp:2026-07-22 12:01:51.38062182 +0000 UTC m=+0.402643514,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-107.ec2.internal,}"
Jul 22 12:01:51.383606 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.383590    2584 policy_none.go:47] "Start"
Jul 22 12:01:51.425154 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.424990    2584 manager.go:337] "Starting Device Plugin manager"
Jul 22 12:01:51.425236 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.425175    2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 12:01:51.425236 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.425191    2584 server.go:87] "Starting device plugin registration server"
Jul 22 12:01:51.425493 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.425481    2584 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 12:01:51.425527 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.425495    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 12:01:51.425654 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.425614    2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 12:01:51.425746 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.425728    2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 12:01:51.425746 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.425745    2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 12:01:51.426225 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.426206    2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 12:01:51.426317 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.426261    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:51.517165 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.517126    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 12:01:51.518425 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.518408    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 12:01:51.518504 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.518475    2584 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 12:01:51.518564 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.518516    2584 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 12:01:51.518617 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.518561    2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 12:01:51.520872 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.520852    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:01:51.525980 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.525940    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:51.526776 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.526760    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:51.526823 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.526797    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:51.526823 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.526811    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:51.526901 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.526841    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.536735 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.536720    2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.536795 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.536742    2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-107.ec2.internal\": node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:51.555687 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.555662    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:51.620626 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.620591    2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-107.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal"]
Jul 22 12:01:51.620759 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.620664    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:51.621607 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.621588    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:51.621705 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.621626    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:51.621705 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.621646    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:51.622974 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.622957    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:51.623102 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.623079    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.623151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.623109    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:51.623769 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.623753    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:51.623858 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.623789    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:51.623858 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.623804    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:51.623968 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.623804    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:51.623968 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.623953    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:51.623968 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.623969    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:51.625011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.624994    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.625093 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.625028    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:51.625829 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.625813    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:51.625899 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.625843    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:51.625899 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.625861    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:51.656044 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.656018    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:51.660648 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.660628    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/acd362fd4cb61d67b387c384f06f0cd7-config\") pod \"kube-apiserver-proxy-ip-10-0-143-107.ec2.internal\" (UID: \"acd362fd4cb61d67b387c384f06f0cd7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.660730 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.660665    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27862815abc3cf524c8a6913e8194350-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal\" (UID: \"27862815abc3cf524c8a6913e8194350\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.660730 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.660683    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27862815abc3cf524c8a6913e8194350-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal\" (UID: \"27862815abc3cf524c8a6913e8194350\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.666299 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.666281    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-107.ec2.internal\" not found" node="ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.670937 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.670919    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-107.ec2.internal\" not found" node="ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.757189 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.757152    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:51.761993 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.761975    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27862815abc3cf524c8a6913e8194350-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal\" (UID: \"27862815abc3cf524c8a6913e8194350\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.762052 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.762002    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/acd362fd4cb61d67b387c384f06f0cd7-config\") pod \"kube-apiserver-proxy-ip-10-0-143-107.ec2.internal\" (UID: \"acd362fd4cb61d67b387c384f06f0cd7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.762052 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.762021    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27862815abc3cf524c8a6913e8194350-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal\" (UID: \"27862815abc3cf524c8a6913e8194350\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.762130 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.762058    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/acd362fd4cb61d67b387c384f06f0cd7-config\") pod \"kube-apiserver-proxy-ip-10-0-143-107.ec2.internal\" (UID: \"acd362fd4cb61d67b387c384f06f0cd7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.762130 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.762072    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27862815abc3cf524c8a6913e8194350-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal\" (UID: \"27862815abc3cf524c8a6913e8194350\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.762130 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.762109    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27862815abc3cf524c8a6913e8194350-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal\" (UID: \"27862815abc3cf524c8a6913e8194350\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.857764 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.857680    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:51.960758 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:51.960722    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:51.968604 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.968589    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:51.975813 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:51.975262    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:52.060860 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:52.060828    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:52.161101 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:52.161017    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:52.195213 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.195172    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:01:52.262857 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:52.262825    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:52.299957 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.299931    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 12:01:52.300357 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.300082    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 12:01:52.300357 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.300118    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 12:01:52.363018 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:52.362985    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:52.373341 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.373312    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 11:56:51 +0000 UTC" deadline="2028-02-06 14:13:34.025102127 +0000 UTC"
Jul 22 12:01:52.373341 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.373336    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13538h11m41.651769159s"
Jul 22 12:01:52.463731 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:52.463654    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-107.ec2.internal\" not found"
Jul 22 12:01:52.465335 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.465314    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:01:52.558221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.558194    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:52.574086 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.574063    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 12:01:52.576036 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.576018    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-107.ec2.internal"
Jul 22 12:01:52.584491 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.584473    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 12:01:52.584601 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.584584    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:01:52.609934 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:52.609900    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27862815abc3cf524c8a6913e8194350.slice/crio-87849d9b1491b8d0022ddaea36abf32ad7d1bbdbee1631e192e6057d17b53baa WatchSource:0}: Error finding container 87849d9b1491b8d0022ddaea36abf32ad7d1bbdbee1631e192e6057d17b53baa: Status 404 returned error can't find the container with id 87849d9b1491b8d0022ddaea36abf32ad7d1bbdbee1631e192e6057d17b53baa
Jul 22 12:01:52.615226 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:52.615210    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:01:52.936981 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:52.936955    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd362fd4cb61d67b387c384f06f0cd7.slice/crio-e3eac5ae0eb121a6a8437a281bd0e039a549bd6f018a3889255038ce8c074b7e WatchSource:0}: Error finding container e3eac5ae0eb121a6a8437a281bd0e039a549bd6f018a3889255038ce8c074b7e: Status 404 returned error can't find the container with id e3eac5ae0eb121a6a8437a281bd0e039a549bd6f018a3889255038ce8c074b7e
Jul 22 12:01:53.340507 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.340375    2584 apiserver.go:52] "Watching apiserver"
Jul 22 12:01:53.349257 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.349225    2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 12:01:53.349627 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.349604    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-pmljl","openshift-cluster-node-tuning-operator/tuned-7l5c5","openshift-dns/node-resolver-shlxl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal","openshift-multus/multus-6vjhd","openshift-multus/network-metrics-daemon-zkjd8","openshift-network-diagnostics/network-check-target-48x6v","openshift-network-operator/iptables-alerter-l6kj2","kube-system/global-pull-secret-syncer-kds22","kube-system/kube-apiserver-proxy-ip-10-0-143-107.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr","openshift-image-registry/node-ca-7pdv9","openshift-multus/multus-additional-cni-plugins-jgbkp","openshift-ovn-kubernetes/ovnkube-node-hcr2s"]
Jul 22 12:01:53.351429 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.351412    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:01:53.353306 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.353289    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.354396 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.354370    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 12:01:53.354396 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.354370    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 12:01:53.354609 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.354543    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kk7cg\""
Jul 22 12:01:53.354609 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.354594    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-shlxl"
Jul 22 12:01:53.355804 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.355785    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.356220 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.356203    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 12:01:53.356312 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.356239    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2gl6f\""
Jul 22 12:01:53.356607 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.356593    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 12:01:53.357206 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.357164    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 12:01:53.357297 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.357225    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:53.357355 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.357293    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:01:53.357481 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.357464    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hpjpx\""
Jul 22 12:01:53.357529 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.357482    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 12:01:53.358287 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.358266    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 12:01:53.358688 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.358456    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:01:53.358688 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.358525    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 12:01:53.358688 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.358524    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:01:53.358688 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.358569    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 12:01:53.359210 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.358717    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 12:01:53.359210 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.358872    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rrgw5\""
Jul 22 12:01:53.359210 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.359035    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 12:01:53.359779 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.359762    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l6kj2"
Jul 22 12:01:53.362098 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.362080    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:53.362196 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.362125    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:01:53.362342 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.362325    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 12:01:53.362443 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.362359    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mmqng\""
Jul 22 12:01:53.362511 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.362452    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 12:01:53.362511 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.362469    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 12:01:53.363676 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.363656    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.365058 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.365039    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7pdv9"
Jul 22 12:01:53.365911 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.365892    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 12:01:53.366234 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.366199    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 12:01:53.366489 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.366472    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.366589 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.366501    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6qqct\""
Jul 22 12:01:53.367418 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.367396    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 12:01:53.367678 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.367663    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 12:01:53.367919 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.367905    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.368771 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.368749    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 12:01:53.368874 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.368769    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 12:01:53.368874 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.368793    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7c9bk\""
Jul 22 12:01:53.369573 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.369557    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vbzpc\""
Jul 22 12:01:53.369930 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.369913    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 12:01:53.370008 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.369939    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 12:01:53.370265 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370241    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/117cd51d-630f-41b4-9a93-bcf26234caca-tmp\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.370344 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370328    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 12:01:53.370387 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370300    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47gmc\" (UniqueName: \"kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc\") pod \"network-check-target-48x6v\" (UID: \"f308d47a-3937-4b78-8dd1-199f13817de1\") " pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:01:53.370387 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370333    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 12:01:53.370509 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370390    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-system-cni-dir\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.370509 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370447    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-os-release\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.370509 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370490    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-conf-dir\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.370509 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370496    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 12:01:53.370714 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370517    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-daemon-config\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.370714 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370527    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 12:01:53.370714 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370543    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/864f6b9f-0aed-44a8-a12c-35f051e293e3-iptables-alerter-script\") pod \"iptables-alerter-l6kj2\" (UID: \"864f6b9f-0aed-44a8-a12c-35f051e293e3\") " pod="openshift-network-operator/iptables-alerter-l6kj2"
Jul 22 12:01:53.370714 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370558    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dhxbd\""
Jul 22 12:01:53.370714 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370575    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74pdd\" (UniqueName: \"kubernetes.io/projected/117cd51d-630f-41b4-9a93-bcf26234caca-kube-api-access-74pdd\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.370714 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370614    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dfea4d2-cebc-4c94-b4ac-3641e198d0e9-tmp-dir\") pod \"node-resolver-shlxl\" (UID: \"1dfea4d2-cebc-4c94-b4ac-3641e198d0e9\") " pod="openshift-dns/node-resolver-shlxl"
Jul 22 12:01:53.370714 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370674    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntw4m\" (UniqueName: \"kubernetes.io/projected/1dfea4d2-cebc-4c94-b4ac-3641e198d0e9-kube-api-access-ntw4m\") pod \"node-resolver-shlxl\" (UID: \"1dfea4d2-cebc-4c94-b4ac-3641e198d0e9\") " pod="openshift-dns/node-resolver-shlxl"
Jul 22 12:01:53.371085 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370734    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-cni-dir\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.371085 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370764    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-var-lib-cni-bin\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.371085 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370809    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z4x7\" (UniqueName: \"kubernetes.io/projected/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-kube-api-access-9z4x7\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.371085 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370871    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:53.371085 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.370976    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1dfea4d2-cebc-4c94-b4ac-3641e198d0e9-hosts-file\") pod \"node-resolver-shlxl\" (UID: \"1dfea4d2-cebc-4c94-b4ac-3641e198d0e9\") " pod="openshift-dns/node-resolver-shlxl"
Jul 22 12:01:53.371085 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371002    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-socket-dir-parent\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.371085 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371035    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-run-k8s-cni-cncf-io\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371061    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rcvc\" (UniqueName: \"kubernetes.io/projected/34b5faa4-567b-421f-a050-87e201154841-kube-api-access-8rcvc\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371120    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spr8g\" (UniqueName: \"kubernetes.io/projected/864f6b9f-0aed-44a8-a12c-35f051e293e3-kube-api-access-spr8g\") pod \"iptables-alerter-l6kj2\" (UID: \"864f6b9f-0aed-44a8-a12c-35f051e293e3\") " pod="openshift-network-operator/iptables-alerter-l6kj2"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371148    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-modprobe-d\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371182    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-sysctl-conf\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371203    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-sys\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371228    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-lib-modules\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371244    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-cnibin\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371258    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-hostroot\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371278    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-run\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371300    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-host\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371324    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/117cd51d-630f-41b4-9a93-bcf26234caca-etc-tuned\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371350    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-sysctl-d\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371375    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/464a2ed7-bcdb-4426-b488-9d826b166405-agent-certs\") pod \"konnectivity-agent-pmljl\" (UID: \"464a2ed7-bcdb-4426-b488-9d826b166405\") " pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371416    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-cni-binary-copy\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371438    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-var-lib-cni-multus\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.371444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371474    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/864f6b9f-0aed-44a8-a12c-35f051e293e3-host-slash\") pod \"iptables-alerter-l6kj2\" (UID: \"864f6b9f-0aed-44a8-a12c-35f051e293e3\") " pod="openshift-network-operator/iptables-alerter-l6kj2"
Jul 22 12:01:53.372304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371533    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-sysconfig\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.372304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371549    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-systemd\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.372304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371563    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-var-lib-kubelet\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.372304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371603    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/464a2ed7-bcdb-4426-b488-9d826b166405-konnectivity-ca\") pod \"konnectivity-agent-pmljl\" (UID: \"464a2ed7-bcdb-4426-b488-9d826b166405\") " pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:01:53.372304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.371886    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-etc-kubernetes\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.372304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.372163    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-kubernetes\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.372304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.372231    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-run-netns\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.372304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.372274    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-var-lib-kubelet\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.372741 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.372308    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-run-multus-certs\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.372741 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.372559    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 12:01:53.375340 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.375320    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 12:01:53.392935 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.392903    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 12:01:53.432951 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.432919    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5xltt"
Jul 22 12:01:53.442583 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.442557    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5xltt"
Jul 22 12:01:53.459955 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.459918    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 12:01:53.472525 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472495    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z4x7\" (UniqueName: \"kubernetes.io/projected/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-kube-api-access-9z4x7\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.472667 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472538    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:53.472667 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472571    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rcvc\" (UniqueName: \"kubernetes.io/projected/34b5faa4-567b-421f-a050-87e201154841-kube-api-access-8rcvc\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:53.472667 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472603    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-cni-bin\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.472667 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472630    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-os-release\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.472667 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472657    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-socket-dir-parent\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.472932 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.472680    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:53.472932 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472688    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-run-k8s-cni-cncf-io\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.472932 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472738    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee8865d5-4008-4275-9bd2-fb89c607064b-env-overrides\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.472932 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.472771    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs podName:34b5faa4-567b-421f-a050-87e201154841 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:53.972745566 +0000 UTC m=+2.994767274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs") pod "network-metrics-daemon-zkjd8" (UID: "34b5faa4-567b-421f-a050-87e201154841") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:53.472932 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472825    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eef02bfb-180d-49b0-a4c6-52cf46557c5c-serviceca\") pod \"node-ca-7pdv9\" (UID: \"eef02bfb-180d-49b0-a4c6-52cf46557c5c\") " pod="openshift-image-registry/node-ca-7pdv9"
Jul 22 12:01:53.472932 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472834    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-socket-dir-parent\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.472932 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472864    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-sysctl-conf\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.472932 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472852    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-run-k8s-cni-cncf-io\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.472932 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472899    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-sys\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.472932 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472925    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-cnibin\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472952    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-etc-openvswitch\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.472979    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473006    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p85r2\" (UniqueName: \"kubernetes.io/projected/eef02bfb-180d-49b0-a4c6-52cf46557c5c-kube-api-access-p85r2\") pod \"node-ca-7pdv9\" (UID: \"eef02bfb-180d-49b0-a4c6-52cf46557c5c\") " pod="openshift-image-registry/node-ca-7pdv9"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473016    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-sys\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473031    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-run\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473034    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-sysctl-conf\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473038    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-cnibin\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473057    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-sysctl-d\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473107    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/464a2ed7-bcdb-4426-b488-9d826b166405-agent-certs\") pod \"konnectivity-agent-pmljl\" (UID: \"464a2ed7-bcdb-4426-b488-9d826b166405\") " pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473109    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-run\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473135    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-cni-binary-copy\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473198    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-var-lib-cni-multus\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473225    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/864f6b9f-0aed-44a8-a12c-35f051e293e3-host-slash\") pod \"iptables-alerter-l6kj2\" (UID: \"864f6b9f-0aed-44a8-a12c-35f051e293e3\") " pod="openshift-network-operator/iptables-alerter-l6kj2"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473252    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-sysctl-d\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473256    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-run-netns\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473274    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-var-lib-cni-multus\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.473414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473305    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-var-lib-openvswitch\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473294    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/864f6b9f-0aed-44a8-a12c-35f051e293e3-host-slash\") pod \"iptables-alerter-l6kj2\" (UID: \"864f6b9f-0aed-44a8-a12c-35f051e293e3\") " pod="openshift-network-operator/iptables-alerter-l6kj2"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473348    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-device-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473407    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-sysconfig\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473434    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-var-lib-kubelet\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473494    2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473519    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-var-lib-kubelet\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473460    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/464a2ed7-bcdb-4426-b488-9d826b166405-konnectivity-ca\") pod \"konnectivity-agent-pmljl\" (UID: \"464a2ed7-bcdb-4426-b488-9d826b166405\") " pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473582    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-sysconfig\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473616    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-slash\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473650    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473753    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473778    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-cni-binary-copy\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473794    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-sys-fs\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473855    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-kubernetes\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473880    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-run-netns\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473941    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-run-netns\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.474219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473951    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-kubernetes\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.473987    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-var-lib-kubelet\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474018    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474048    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/507d1473-a97c-4fe5-9829-4f7fbefc0967-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474058    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-var-lib-kubelet\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474065    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/464a2ed7-bcdb-4426-b488-9d826b166405-konnectivity-ca\") pod \"konnectivity-agent-pmljl\" (UID: \"464a2ed7-bcdb-4426-b488-9d826b166405\") " pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474087    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcxjc\" (UniqueName: \"kubernetes.io/projected/507d1473-a97c-4fe5-9829-4f7fbefc0967-kube-api-access-bcxjc\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474122    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/117cd51d-630f-41b4-9a93-bcf26234caca-tmp\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474150    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47gmc\" (UniqueName: \"kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc\") pod \"network-check-target-48x6v\" (UID: \"f308d47a-3937-4b78-8dd1-199f13817de1\") " pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474178    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-system-cni-dir\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-os-release\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474230    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-conf-dir\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474257    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-cni-netd\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474267    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-system-cni-dir\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474287    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee8865d5-4008-4275-9bd2-fb89c607064b-ovnkube-config\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474313    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-cnibin\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474316    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-conf-dir\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.475000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474341    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntw4m\" (UniqueName: \"kubernetes.io/projected/1dfea4d2-cebc-4c94-b4ac-3641e198d0e9-kube-api-access-ntw4m\") pod \"node-resolver-shlxl\" (UID: \"1dfea4d2-cebc-4c94-b4ac-3641e198d0e9\") " pod="openshift-dns/node-resolver-shlxl"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474368    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-var-lib-cni-bin\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474376    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-os-release\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474391    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-run-ovn\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474433    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee8865d5-4008-4275-9bd2-fb89c607064b-ovn-node-metrics-cert\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474463    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/507d1473-a97c-4fe5-9829-4f7fbefc0967-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474479    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-var-lib-cni-bin\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474545    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1dfea4d2-cebc-4c94-b4ac-3641e198d0e9-hosts-file\") pod \"node-resolver-shlxl\" (UID: \"1dfea4d2-cebc-4c94-b4ac-3641e198d0e9\") " pod="openshift-dns/node-resolver-shlxl"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474578    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spr8g\" (UniqueName: \"kubernetes.io/projected/864f6b9f-0aed-44a8-a12c-35f051e293e3-kube-api-access-spr8g\") pod \"iptables-alerter-l6kj2\" (UID: \"864f6b9f-0aed-44a8-a12c-35f051e293e3\") " pod="openshift-network-operator/iptables-alerter-l6kj2"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474608    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-run-openvswitch\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474618    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1dfea4d2-cebc-4c94-b4ac-3641e198d0e9-hosts-file\") pod \"node-resolver-shlxl\" (UID: \"1dfea4d2-cebc-4c94-b4ac-3641e198d0e9\") " pod="openshift-dns/node-resolver-shlxl"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474635    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-socket-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474661    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bvtw\" (UniqueName: \"kubernetes.io/projected/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-kube-api-access-7bvtw\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474717    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-modprobe-d\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474761    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-lib-modules\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474786    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-hostroot\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474814    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f78d8d30-61ed-4179-a3d4-bd7188b22aab-kubelet-config\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:53.475886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474841    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-registration-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474864    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eef02bfb-180d-49b0-a4c6-52cf46557c5c-host\") pod \"node-ca-7pdv9\" (UID: \"eef02bfb-180d-49b0-a4c6-52cf46557c5c\") " pod="openshift-image-registry/node-ca-7pdv9"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474869    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-hostroot\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474891    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-host\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474916    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/117cd51d-630f-41b4-9a93-bcf26234caca-etc-tuned\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474933    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-modprobe-d\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474942    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-run-systemd\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474936    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-lib-modules\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474956    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-host\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.474979    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee8865d5-4008-4275-9bd2-fb89c607064b-ovnkube-script-lib\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475010    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-system-cni-dir\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475036    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-kubelet\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475061    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-log-socket\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475087    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-systemd\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475113    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-etc-kubernetes\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475141    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88sjs\" (UniqueName: \"kubernetes.io/projected/ee8865d5-4008-4275-9bd2-fb89c607064b-kube-api-access-88sjs\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475168    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f78d8d30-61ed-4179-a3d4-bd7188b22aab-dbus\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:53.476744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475178    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-etc-kubernetes\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475191    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475181    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/117cd51d-630f-41b4-9a93-bcf26234caca-etc-systemd\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475213    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-run-multus-certs\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475241    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-systemd-units\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475247    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-host-run-multus-certs\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475271    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-node-log\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475297    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/507d1473-a97c-4fe5-9829-4f7fbefc0967-cni-binary-copy\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475333    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-etc-selinux\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475376    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-daemon-config\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475419    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/864f6b9f-0aed-44a8-a12c-35f051e293e3-iptables-alerter-script\") pod \"iptables-alerter-l6kj2\" (UID: \"864f6b9f-0aed-44a8-a12c-35f051e293e3\") " pod="openshift-network-operator/iptables-alerter-l6kj2"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475444    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74pdd\" (UniqueName: \"kubernetes.io/projected/117cd51d-630f-41b4-9a93-bcf26234caca-kube-api-access-74pdd\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475499    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dfea4d2-cebc-4c94-b4ac-3641e198d0e9-tmp-dir\") pod \"node-resolver-shlxl\" (UID: \"1dfea4d2-cebc-4c94-b4ac-3641e198d0e9\") " pod="openshift-dns/node-resolver-shlxl"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475525    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-cni-dir\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475696    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-cni-dir\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475843    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dfea4d2-cebc-4c94-b4ac-3641e198d0e9-tmp-dir\") pod \"node-resolver-shlxl\" (UID: \"1dfea4d2-cebc-4c94-b4ac-3641e198d0e9\") " pod="openshift-dns/node-resolver-shlxl"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.475958    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-multus-daemon-config\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.477528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.476071    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/864f6b9f-0aed-44a8-a12c-35f051e293e3-iptables-alerter-script\") pod \"iptables-alerter-l6kj2\" (UID: \"864f6b9f-0aed-44a8-a12c-35f051e293e3\") " pod="openshift-network-operator/iptables-alerter-l6kj2"
Jul 22 12:01:53.478400 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.477584    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/117cd51d-630f-41b4-9a93-bcf26234caca-tmp\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.478400 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.477850    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/117cd51d-630f-41b4-9a93-bcf26234caca-etc-tuned\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.478400 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.477989    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/464a2ed7-bcdb-4426-b488-9d826b166405-agent-certs\") pod \"konnectivity-agent-pmljl\" (UID: \"464a2ed7-bcdb-4426-b488-9d826b166405\") " pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:01:53.483889 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.483864    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:01:53.483990 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.483895    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:01:53.483990 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.483910    2584 projected.go:196] Error preparing data for projected volume kube-api-access-47gmc for pod openshift-network-diagnostics/network-check-target-48x6v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:53.483990 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.483975    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc podName:f308d47a-3937-4b78-8dd1-199f13817de1 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:53.983956996 +0000 UTC m=+3.005978695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-47gmc" (UniqueName: "kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc") pod "network-check-target-48x6v" (UID: "f308d47a-3937-4b78-8dd1-199f13817de1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:53.487057 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.487035    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z4x7\" (UniqueName: \"kubernetes.io/projected/d39fb4a7-d50d-4ec1-8af1-a424fd0a453f-kube-api-access-9z4x7\") pod \"multus-6vjhd\" (UID: \"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f\") " pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.487604 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.487577    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rcvc\" (UniqueName: \"kubernetes.io/projected/34b5faa4-567b-421f-a050-87e201154841-kube-api-access-8rcvc\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:53.488319 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.488292    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntw4m\" (UniqueName: \"kubernetes.io/projected/1dfea4d2-cebc-4c94-b4ac-3641e198d0e9-kube-api-access-ntw4m\") pod \"node-resolver-shlxl\" (UID: \"1dfea4d2-cebc-4c94-b4ac-3641e198d0e9\") " pod="openshift-dns/node-resolver-shlxl"
Jul 22 12:01:53.488757 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.488733    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74pdd\" (UniqueName: \"kubernetes.io/projected/117cd51d-630f-41b4-9a93-bcf26234caca-kube-api-access-74pdd\") pod \"tuned-7l5c5\" (UID: \"117cd51d-630f-41b4-9a93-bcf26234caca\") " pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.488919 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.488902    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spr8g\" (UniqueName: \"kubernetes.io/projected/864f6b9f-0aed-44a8-a12c-35f051e293e3-kube-api-access-spr8g\") pod \"iptables-alerter-l6kj2\" (UID: \"864f6b9f-0aed-44a8-a12c-35f051e293e3\") " pod="openshift-network-operator/iptables-alerter-l6kj2"
Jul 22 12:01:53.523549 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.523493    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal" event={"ID":"27862815abc3cf524c8a6913e8194350","Type":"ContainerStarted","Data":"87849d9b1491b8d0022ddaea36abf32ad7d1bbdbee1631e192e6057d17b53baa"}
Jul 22 12:01:53.524892 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.524865    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-107.ec2.internal" event={"ID":"acd362fd4cb61d67b387c384f06f0cd7","Type":"ContainerStarted","Data":"e3eac5ae0eb121a6a8437a281bd0e039a549bd6f018a3889255038ce8c074b7e"}
Jul 22 12:01:53.576497 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.576447    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:53.576497 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.576502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/507d1473-a97c-4fe5-9829-4f7fbefc0967-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.576765 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.576541    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcxjc\" (UniqueName: \"kubernetes.io/projected/507d1473-a97c-4fe5-9829-4f7fbefc0967-kube-api-access-bcxjc\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.576765 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.576641    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:53.576765 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.576733    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret podName:f78d8d30-61ed-4179-a3d4-bd7188b22aab nodeName:}" failed. No retries permitted until 2026-07-22 12:01:54.076712274 +0000 UTC m=+3.098733958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret") pod "global-pull-secret-syncer-kds22" (UID: "f78d8d30-61ed-4179-a3d4-bd7188b22aab") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:53.576950 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.576844    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-cni-netd\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.576950 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.576886    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee8865d5-4008-4275-9bd2-fb89c607064b-ovnkube-config\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.576950 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.576913    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-cnibin\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.576950 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.576944    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-run-ovn\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.577153 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.576970    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-cni-netd\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.577153 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.576979    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee8865d5-4008-4275-9bd2-fb89c607064b-ovn-node-metrics-cert\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.577153 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577030    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/507d1473-a97c-4fe5-9829-4f7fbefc0967-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.577153 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577064    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-run-openvswitch\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.577153 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577090    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-socket-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.577153 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577117    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bvtw\" (UniqueName: \"kubernetes.io/projected/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-kube-api-access-7bvtw\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.577153 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577147    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f78d8d30-61ed-4179-a3d4-bd7188b22aab-kubelet-config\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577173    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-registration-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577183    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/507d1473-a97c-4fe5-9829-4f7fbefc0967-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577200    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eef02bfb-180d-49b0-a4c6-52cf46557c5c-host\") pod \"node-ca-7pdv9\" (UID: \"eef02bfb-180d-49b0-a4c6-52cf46557c5c\") " pod="openshift-image-registry/node-ca-7pdv9"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577228    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-run-systemd\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577295    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-cnibin\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577311    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eef02bfb-180d-49b0-a4c6-52cf46557c5c-host\") pod \"node-ca-7pdv9\" (UID: \"eef02bfb-180d-49b0-a4c6-52cf46557c5c\") " pod="openshift-image-registry/node-ca-7pdv9"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577363    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f78d8d30-61ed-4179-a3d4-bd7188b22aab-kubelet-config\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577437    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee8865d5-4008-4275-9bd2-fb89c607064b-ovnkube-script-lib\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577493    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-system-cni-dir\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577529    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-kubelet\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577547    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-run-ovn\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577557    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-log-socket\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.577610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577590    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-run-systemd\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577643    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-run-openvswitch\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577698    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-socket-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577708    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-registration-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577752    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-kubelet\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577825    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-log-socket\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577881    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-system-cni-dir\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577920    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88sjs\" (UniqueName: \"kubernetes.io/projected/ee8865d5-4008-4275-9bd2-fb89c607064b-kube-api-access-88sjs\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577946    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f78d8d30-61ed-4179-a3d4-bd7188b22aab-dbus\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577949    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/507d1473-a97c-4fe5-9829-4f7fbefc0967-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.577969    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578000    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-systemd-units\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578016    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-node-log\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578032    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/507d1473-a97c-4fe5-9829-4f7fbefc0967-cni-binary-copy\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578036    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f78d8d30-61ed-4179-a3d4-bd7188b22aab-dbus\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578074    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-node-log\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578082    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-etc-selinux\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.578263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578131    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-systemd-units\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578181    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee8865d5-4008-4275-9bd2-fb89c607064b-ovnkube-config\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578267    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578287    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee8865d5-4008-4275-9bd2-fb89c607064b-ovnkube-script-lib\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578323    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-etc-selinux\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578407    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-cni-bin\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578443    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-os-release\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578499    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-cni-bin\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578569    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/507d1473-a97c-4fe5-9829-4f7fbefc0967-os-release\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578618    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee8865d5-4008-4275-9bd2-fb89c607064b-env-overrides\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578663    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eef02bfb-180d-49b0-a4c6-52cf46557c5c-serviceca\") pod \"node-ca-7pdv9\" (UID: \"eef02bfb-180d-49b0-a4c6-52cf46557c5c\") " pod="openshift-image-registry/node-ca-7pdv9"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578715    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/507d1473-a97c-4fe5-9829-4f7fbefc0967-cni-binary-copy\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578793    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-etc-openvswitch\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578813    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578829    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p85r2\" (UniqueName: \"kubernetes.io/projected/eef02bfb-180d-49b0-a4c6-52cf46557c5c-kube-api-access-p85r2\") pod \"node-ca-7pdv9\" (UID: \"eef02bfb-180d-49b0-a4c6-52cf46557c5c\") " pod="openshift-image-registry/node-ca-7pdv9"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578858    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-run-netns\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578877    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-var-lib-openvswitch\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578892    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-device-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578912    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-slash\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578934    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578955    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578973    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-sys-fs\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.578999    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-run-netns\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.579044    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-sys-fs\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.579055    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-etc-openvswitch\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.579074    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-var-lib-openvswitch\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.579072    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee8865d5-4008-4275-9bd2-fb89c607064b-env-overrides\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.579116    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.579126    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-device-dir\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.579164    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.579171    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.579173    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eef02bfb-180d-49b0-a4c6-52cf46557c5c-serviceca\") pod \"node-ca-7pdv9\" (UID: \"eef02bfb-180d-49b0-a4c6-52cf46557c5c\") " pod="openshift-image-registry/node-ca-7pdv9"
Jul 22 12:01:53.579962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.579197    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee8865d5-4008-4275-9bd2-fb89c607064b-host-slash\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.580975 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.580952    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee8865d5-4008-4275-9bd2-fb89c607064b-ovn-node-metrics-cert\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.592277 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.592198    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bvtw\" (UniqueName: \"kubernetes.io/projected/8115d8cc-5a84-43e3-8e8d-ce07ad68475d-kube-api-access-7bvtw\") pod \"aws-ebs-csi-driver-node-jkvxr\" (UID: \"8115d8cc-5a84-43e3-8e8d-ce07ad68475d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.592486 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.592460    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcxjc\" (UniqueName: \"kubernetes.io/projected/507d1473-a97c-4fe5-9829-4f7fbefc0967-kube-api-access-bcxjc\") pod \"multus-additional-cni-plugins-jgbkp\" (UID: \"507d1473-a97c-4fe5-9829-4f7fbefc0967\") " pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.592949 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.592925    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p85r2\" (UniqueName: \"kubernetes.io/projected/eef02bfb-180d-49b0-a4c6-52cf46557c5c-kube-api-access-p85r2\") pod \"node-ca-7pdv9\" (UID: \"eef02bfb-180d-49b0-a4c6-52cf46557c5c\") " pod="openshift-image-registry/node-ca-7pdv9"
Jul 22 12:01:53.593416 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.593392    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88sjs\" (UniqueName: \"kubernetes.io/projected/ee8865d5-4008-4275-9bd2-fb89c607064b-kube-api-access-88sjs\") pod \"ovnkube-node-hcr2s\" (UID: \"ee8865d5-4008-4275-9bd2-fb89c607064b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.660573 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.660545    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:01:53.661697 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.661672    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:01:53.668931 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.668908    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7l5c5"
Jul 22 12:01:53.671306 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:53.671278    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod464a2ed7_bcdb_4426_b488_9d826b166405.slice/crio-4e04e20d1ce7d566e50845699e0829f49519730be53b02c735848d9548a80e6b WatchSource:0}: Error finding container 4e04e20d1ce7d566e50845699e0829f49519730be53b02c735848d9548a80e6b: Status 404 returned error can't find the container with id 4e04e20d1ce7d566e50845699e0829f49519730be53b02c735848d9548a80e6b
Jul 22 12:01:53.678370 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.678334    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-shlxl"
Jul 22 12:01:53.679660 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:53.679635    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117cd51d_630f_41b4_9a93_bcf26234caca.slice/crio-cc022424a2979ae97186ac34ae8408facbfc9cbfcfb27a4f5286c08231e92196 WatchSource:0}: Error finding container cc022424a2979ae97186ac34ae8408facbfc9cbfcfb27a4f5286c08231e92196: Status 404 returned error can't find the container with id cc022424a2979ae97186ac34ae8408facbfc9cbfcfb27a4f5286c08231e92196
Jul 22 12:01:53.684359 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.684338    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6vjhd"
Jul 22 12:01:53.687985 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:53.687958    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dfea4d2_cebc_4c94_b4ac_3641e198d0e9.slice/crio-f8b23179a49fc5f6389bb57fa59192483e51f168e91d30dbf400374bc95a824a WatchSource:0}: Error finding container f8b23179a49fc5f6389bb57fa59192483e51f168e91d30dbf400374bc95a824a: Status 404 returned error can't find the container with id f8b23179a49fc5f6389bb57fa59192483e51f168e91d30dbf400374bc95a824a
Jul 22 12:01:53.691460 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.691427    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l6kj2"
Jul 22 12:01:53.693634 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:53.693609    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39fb4a7_d50d_4ec1_8af1_a424fd0a453f.slice/crio-e13be184c2d57f1cecffdd89e601c98e186273390cdb50c36f811e87243882c3 WatchSource:0}: Error finding container e13be184c2d57f1cecffdd89e601c98e186273390cdb50c36f811e87243882c3: Status 404 returned error can't find the container with id e13be184c2d57f1cecffdd89e601c98e186273390cdb50c36f811e87243882c3
Jul 22 12:01:53.699799 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.699775    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr"
Jul 22 12:01:53.701625 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:53.701591    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864f6b9f_0aed_44a8_a12c_35f051e293e3.slice/crio-12b2a8d93c725a344cce5308a9977db49de9ce1512ab944bbbfcf77c8e6f2422 WatchSource:0}: Error finding container 12b2a8d93c725a344cce5308a9977db49de9ce1512ab944bbbfcf77c8e6f2422: Status 404 returned error can't find the container with id 12b2a8d93c725a344cce5308a9977db49de9ce1512ab944bbbfcf77c8e6f2422
Jul 22 12:01:53.707935 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.707910    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7pdv9"
Jul 22 12:01:53.708716 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:53.708689    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8115d8cc_5a84_43e3_8e8d_ce07ad68475d.slice/crio-f8a22c53e66afc1c7929db701fc087e66605a514199920574cc4258f0b6aecab WatchSource:0}: Error finding container f8a22c53e66afc1c7929db701fc087e66605a514199920574cc4258f0b6aecab: Status 404 returned error can't find the container with id f8a22c53e66afc1c7929db701fc087e66605a514199920574cc4258f0b6aecab
Jul 22 12:01:53.716264 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.716238    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jgbkp"
Jul 22 12:01:53.716575 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:53.716536    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef02bfb_180d_49b0_a4c6_52cf46557c5c.slice/crio-6c3658db428f9b50fa8cf6ae0ae2b321f9f21a90c882575dfb789a0665e54375 WatchSource:0}: Error finding container 6c3658db428f9b50fa8cf6ae0ae2b321f9f21a90c882575dfb789a0665e54375: Status 404 returned error can't find the container with id 6c3658db428f9b50fa8cf6ae0ae2b321f9f21a90c882575dfb789a0665e54375
Jul 22 12:01:53.721357 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.721336    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:01:53.727015 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:53.726987    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod507d1473_a97c_4fe5_9829_4f7fbefc0967.slice/crio-03486e861b9df61d57e0a67e929fc514308622ad1925bb039743d5c0cefebc67 WatchSource:0}: Error finding container 03486e861b9df61d57e0a67e929fc514308622ad1925bb039743d5c0cefebc67: Status 404 returned error can't find the container with id 03486e861b9df61d57e0a67e929fc514308622ad1925bb039743d5c0cefebc67
Jul 22 12:01:53.733313 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:01:53.733283    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8865d5_4008_4275_9bd2_fb89c607064b.slice/crio-c2ca17d0cd2be449e56a0ae5fa84c3153b388c2719ed6e99cfbdcc810e3adb3c WatchSource:0}: Error finding container c2ca17d0cd2be449e56a0ae5fa84c3153b388c2719ed6e99cfbdcc810e3adb3c: Status 404 returned error can't find the container with id c2ca17d0cd2be449e56a0ae5fa84c3153b388c2719ed6e99cfbdcc810e3adb3c
Jul 22 12:01:53.982622 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:53.982545    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:53.982785 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.982731    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:53.982918 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:53.982808    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs podName:34b5faa4-567b-421f-a050-87e201154841 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:54.982786304 +0000 UTC m=+4.004807992 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs") pod "network-metrics-daemon-zkjd8" (UID: "34b5faa4-567b-421f-a050-87e201154841") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:54.083948 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.083905    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:54.084139 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.083965    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47gmc\" (UniqueName: \"kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc\") pod \"network-check-target-48x6v\" (UID: \"f308d47a-3937-4b78-8dd1-199f13817de1\") " pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:01:54.084139 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:54.084112    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:54.084253 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:54.084184    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret podName:f78d8d30-61ed-4179-a3d4-bd7188b22aab nodeName:}" failed. No retries permitted until 2026-07-22 12:01:55.084162664 +0000 UTC m=+4.106184350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret") pod "global-pull-secret-syncer-kds22" (UID: "f78d8d30-61ed-4179-a3d4-bd7188b22aab") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:54.084253 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:54.084184    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:01:54.084253 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:54.084211    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:01:54.084253 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:54.084229    2584 projected.go:196] Error preparing data for projected volume kube-api-access-47gmc for pod openshift-network-diagnostics/network-check-target-48x6v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:54.084494 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:54.084292    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc podName:f308d47a-3937-4b78-8dd1-199f13817de1 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:55.084274492 +0000 UTC m=+4.106296187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-47gmc" (UniqueName: "kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc") pod "network-check-target-48x6v" (UID: "f308d47a-3937-4b78-8dd1-199f13817de1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:54.423596 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.423516    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:01:54.445096 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.445055    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 11:56:53 +0000 UTC" deadline="2028-01-03 00:00:24.266571017 +0000 UTC"
Jul 22 12:01:54.445096 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.445091    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12707h58m29.821484986s"
Jul 22 12:01:54.529541 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.529434    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-shlxl" event={"ID":"1dfea4d2-cebc-4c94-b4ac-3641e198d0e9","Type":"ContainerStarted","Data":"f8b23179a49fc5f6389bb57fa59192483e51f168e91d30dbf400374bc95a824a"}
Jul 22 12:01:54.531188 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.531149    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgbkp" event={"ID":"507d1473-a97c-4fe5-9829-4f7fbefc0967","Type":"ContainerStarted","Data":"03486e861b9df61d57e0a67e929fc514308622ad1925bb039743d5c0cefebc67"}
Jul 22 12:01:54.532574 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.532549    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l6kj2" event={"ID":"864f6b9f-0aed-44a8-a12c-35f051e293e3","Type":"ContainerStarted","Data":"12b2a8d93c725a344cce5308a9977db49de9ce1512ab944bbbfcf77c8e6f2422"}
Jul 22 12:01:54.533879 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.533838    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7l5c5" event={"ID":"117cd51d-630f-41b4-9a93-bcf26234caca","Type":"ContainerStarted","Data":"cc022424a2979ae97186ac34ae8408facbfc9cbfcfb27a4f5286c08231e92196"}
Jul 22 12:01:54.535519 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.535482    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pmljl" event={"ID":"464a2ed7-bcdb-4426-b488-9d826b166405","Type":"ContainerStarted","Data":"4e04e20d1ce7d566e50845699e0829f49519730be53b02c735848d9548a80e6b"}
Jul 22 12:01:54.536936 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.536910    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s" event={"ID":"ee8865d5-4008-4275-9bd2-fb89c607064b","Type":"ContainerStarted","Data":"c2ca17d0cd2be449e56a0ae5fa84c3153b388c2719ed6e99cfbdcc810e3adb3c"}
Jul 22 12:01:54.538498 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.538466    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7pdv9" event={"ID":"eef02bfb-180d-49b0-a4c6-52cf46557c5c","Type":"ContainerStarted","Data":"6c3658db428f9b50fa8cf6ae0ae2b321f9f21a90c882575dfb789a0665e54375"}
Jul 22 12:01:54.539839 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.539778    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr" event={"ID":"8115d8cc-5a84-43e3-8e8d-ce07ad68475d","Type":"ContainerStarted","Data":"f8a22c53e66afc1c7929db701fc087e66605a514199920574cc4258f0b6aecab"}
Jul 22 12:01:54.541140 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.541116    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6vjhd" event={"ID":"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f","Type":"ContainerStarted","Data":"e13be184c2d57f1cecffdd89e601c98e186273390cdb50c36f811e87243882c3"}
Jul 22 12:01:54.992340 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:54.992300    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:54.992550 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:54.992486    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:54.992618 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:54.992560    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs podName:34b5faa4-567b-421f-a050-87e201154841 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:56.992538187 +0000 UTC m=+6.014559881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs") pod "network-metrics-daemon-zkjd8" (UID: "34b5faa4-567b-421f-a050-87e201154841") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:55.093402 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:55.093362    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:55.093617 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:55.093427    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47gmc\" (UniqueName: \"kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc\") pod \"network-check-target-48x6v\" (UID: \"f308d47a-3937-4b78-8dd1-199f13817de1\") " pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:01:55.107584 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:55.107536    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:55.107730 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:55.107649    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret podName:f78d8d30-61ed-4179-a3d4-bd7188b22aab nodeName:}" failed. No retries permitted until 2026-07-22 12:01:57.107611813 +0000 UTC m=+6.129633512 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret") pod "global-pull-secret-syncer-kds22" (UID: "f78d8d30-61ed-4179-a3d4-bd7188b22aab") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:55.108281 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:55.108250    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:01:55.108281 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:55.108278    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:01:55.108478 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:55.108293    2584 projected.go:196] Error preparing data for projected volume kube-api-access-47gmc for pod openshift-network-diagnostics/network-check-target-48x6v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:55.108478 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:55.108353    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc podName:f308d47a-3937-4b78-8dd1-199f13817de1 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:57.108332314 +0000 UTC m=+6.130354010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-47gmc" (UniqueName: "kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc") pod "network-check-target-48x6v" (UID: "f308d47a-3937-4b78-8dd1-199f13817de1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:55.446230 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:55.446158    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 11:56:53 +0000 UTC" deadline="2028-01-29 20:08:27.228299633 +0000 UTC"
Jul 22 12:01:55.446230 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:55.446227    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13352h6m31.782077744s"
Jul 22 12:01:55.522049 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:55.521194    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:55.522049 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:55.521346    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:01:55.522049 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:55.521437    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:55.522049 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:55.521541    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:01:55.522049 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:55.521609    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:01:55.522049 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:55.521680    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:01:56.559142 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:56.558111    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-107.ec2.internal" event={"ID":"acd362fd4cb61d67b387c384f06f0cd7","Type":"ContainerStarted","Data":"e8453768241992b2a285dddaf8af1c6f45cdd6b87c1b4640ec8b5f13e30744f2"}
Jul 22 12:01:56.567476 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:56.567426    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal" event={"ID":"27862815abc3cf524c8a6913e8194350","Type":"ContainerStarted","Data":"1b3dc4c535531c4ee7392b32d7fdcef3636a0e76cb67b0db9253271f1a74722c"}
Jul 22 12:01:56.577819 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:56.577763    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-107.ec2.internal" podStartSLOduration=4.577742424 podStartE2EDuration="4.577742424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:56.576878751 +0000 UTC m=+5.598900487" watchObservedRunningTime="2026-07-22 12:01:56.577742424 +0000 UTC m=+5.599764131"
Jul 22 12:01:57.010215 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:57.010179    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:57.010423 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:57.010332    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:57.010423 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:57.010402    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs podName:34b5faa4-567b-421f-a050-87e201154841 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:01.01038013 +0000 UTC m=+10.032401811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs") pod "network-metrics-daemon-zkjd8" (UID: "34b5faa4-567b-421f-a050-87e201154841") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:57.110539 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:57.110502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:57.110714 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:57.110555    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47gmc\" (UniqueName: \"kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc\") pod \"network-check-target-48x6v\" (UID: \"f308d47a-3937-4b78-8dd1-199f13817de1\") " pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:01:57.110790 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:57.110708    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:57.110790 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:57.110783    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret podName:f78d8d30-61ed-4179-a3d4-bd7188b22aab nodeName:}" failed. No retries permitted until 2026-07-22 12:02:01.110761158 +0000 UTC m=+10.132782860 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret") pod "global-pull-secret-syncer-kds22" (UID: "f78d8d30-61ed-4179-a3d4-bd7188b22aab") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:57.110910 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:57.110711    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:01:57.110910 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:57.110819    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:01:57.110910 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:57.110832    2584 projected.go:196] Error preparing data for projected volume kube-api-access-47gmc for pod openshift-network-diagnostics/network-check-target-48x6v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:57.110910 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:57.110860    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc podName:f308d47a-3937-4b78-8dd1-199f13817de1 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:01.110852608 +0000 UTC m=+10.132874304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-47gmc" (UniqueName: "kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc") pod "network-check-target-48x6v" (UID: "f308d47a-3937-4b78-8dd1-199f13817de1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:57.520618 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:57.519573    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:01:57.520618 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:57.519777    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:01:57.520618 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:57.520194    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:57.520618 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:57.520309    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:01:57.520618 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:57.520500    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:57.520618 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:57.520589    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:01:59.519673 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:59.519587    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:01:59.520100 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:59.519737    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:01:59.520190 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:59.520170    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:01:59.520291 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:59.520267    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:01:59.520487 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:59.520434    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:01:59.520611 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:01:59.520533    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:01:59.582862 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:59.582828    2584 generic.go:356] "Generic (PLEG): container finished" podID="27862815abc3cf524c8a6913e8194350" containerID="1b3dc4c535531c4ee7392b32d7fdcef3636a0e76cb67b0db9253271f1a74722c" exitCode=0
Jul 22 12:01:59.583043 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:01:59.582882    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal" event={"ID":"27862815abc3cf524c8a6913e8194350","Type":"ContainerDied","Data":"1b3dc4c535531c4ee7392b32d7fdcef3636a0e76cb67b0db9253271f1a74722c"}
Jul 22 12:02:01.048433 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:01.048265    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:01.048886 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:01.048445    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:02:01.048886 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:01.048516    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs podName:34b5faa4-567b-421f-a050-87e201154841 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:09.048496417 +0000 UTC m=+18.070518117 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs") pod "network-metrics-daemon-zkjd8" (UID: "34b5faa4-567b-421f-a050-87e201154841") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:02:01.149627 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:01.149572    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:01.149627 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:01.149625    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47gmc\" (UniqueName: \"kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc\") pod \"network-check-target-48x6v\" (UID: \"f308d47a-3937-4b78-8dd1-199f13817de1\") " pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:01.149847 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:01.149743    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:02:01.149847 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:01.149787    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:02:01.149847 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:01.149807    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:02:01.149847 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:01.149821    2584 projected.go:196] Error preparing data for projected volume kube-api-access-47gmc for pod openshift-network-diagnostics/network-check-target-48x6v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:02:01.149847 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:01.149826    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret podName:f78d8d30-61ed-4179-a3d4-bd7188b22aab nodeName:}" failed. No retries permitted until 2026-07-22 12:02:09.149804423 +0000 UTC m=+18.171826115 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret") pod "global-pull-secret-syncer-kds22" (UID: "f78d8d30-61ed-4179-a3d4-bd7188b22aab") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:02:01.150110 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:01.149870    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc podName:f308d47a-3937-4b78-8dd1-199f13817de1 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:09.149855437 +0000 UTC m=+18.171877135 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-47gmc" (UniqueName: "kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc") pod "network-check-target-48x6v" (UID: "f308d47a-3937-4b78-8dd1-199f13817de1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:02:01.521053 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:01.520265    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:01.521053 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:01.520404    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:01.521053 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:01.520457    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:01.521053 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:01.520523    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:01.521053 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:01.520873    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:01.521053 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:01.520961    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:03.520212 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:03.519702    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:03.520212 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:03.519850    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:03.520212 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:03.520205    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:03.520765 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:03.520307    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:03.520765 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:03.520345    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:03.520765 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:03.520480    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:05.519615 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:05.519578    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:05.520064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:05.519576    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:05.520064 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:05.519709    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:05.520064 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:05.519583    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:05.520064 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:05.519792    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:05.520064 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:05.519889    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:07.518970 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:07.518906    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:07.519432 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:07.518915    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:07.519432 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:07.519038    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:07.519432 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:07.519037    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:07.519432 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:07.519165    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:07.519432 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:07.519288    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:09.109672 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:09.109581    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:09.110137 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:09.109756    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:02:09.110137 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:09.109844    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs podName:34b5faa4-567b-421f-a050-87e201154841 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:25.109825321 +0000 UTC m=+34.131847002 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs") pod "network-metrics-daemon-zkjd8" (UID: "34b5faa4-567b-421f-a050-87e201154841") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:02:09.210619 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:09.210584    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:09.210619 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:09.210622    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47gmc\" (UniqueName: \"kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc\") pod \"network-check-target-48x6v\" (UID: \"f308d47a-3937-4b78-8dd1-199f13817de1\") " pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:09.210858 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:09.210754    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:02:09.210858 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:09.210768    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:02:09.210858 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:09.210787    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:02:09.210858 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:09.210796    2584 projected.go:196] Error preparing data for projected volume kube-api-access-47gmc for pod openshift-network-diagnostics/network-check-target-48x6v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:02:09.210858 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:09.210829    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret podName:f78d8d30-61ed-4179-a3d4-bd7188b22aab nodeName:}" failed. No retries permitted until 2026-07-22 12:02:25.210809808 +0000 UTC m=+34.232831509 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret") pod "global-pull-secret-syncer-kds22" (UID: "f78d8d30-61ed-4179-a3d4-bd7188b22aab") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:02:09.210858 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:09.210849    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc podName:f308d47a-3937-4b78-8dd1-199f13817de1 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:25.210839718 +0000 UTC m=+34.232861402 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-47gmc" (UniqueName: "kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc") pod "network-check-target-48x6v" (UID: "f308d47a-3937-4b78-8dd1-199f13817de1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:02:09.519691 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:09.519611    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:09.519691 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:09.519637    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:09.519978 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:09.519611    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:09.519978 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:09.519747    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:09.519978 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:09.519887    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:09.519978 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:09.519968    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:11.520325 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:11.520293    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:11.520845 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:11.520424    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:11.520845 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:11.520531    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:11.520845 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:11.520420    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:11.520845 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:11.520533    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:11.520845 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:11.520637    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:13.519525 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.518949    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:13.519525 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.519009    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:13.519525 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:13.519321    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:13.519525 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.519041    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:13.519525 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:13.519402    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:13.519525 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:13.519487    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:13.613090 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.613063    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal" event={"ID":"27862815abc3cf524c8a6913e8194350","Type":"ContainerStarted","Data":"94e9e8062d07d5b7b350b8fc5ec3c4d6332502dd71c9c7a9004e5e1df911aa30"}
Jul 22 12:02:13.614648 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.614625    2584 generic.go:356] "Generic (PLEG): container finished" podID="507d1473-a97c-4fe5-9829-4f7fbefc0967" containerID="5e6f66851e0f6d040f43d79ee69a2fa3b09de58e9e76e4b86f50036414a9e00e" exitCode=0
Jul 22 12:02:13.614762 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.614697    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgbkp" event={"ID":"507d1473-a97c-4fe5-9829-4f7fbefc0967","Type":"ContainerDied","Data":"5e6f66851e0f6d040f43d79ee69a2fa3b09de58e9e76e4b86f50036414a9e00e"}
Jul 22 12:02:13.616226 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.616189    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7l5c5" event={"ID":"117cd51d-630f-41b4-9a93-bcf26234caca","Type":"ContainerStarted","Data":"4406b15a17fbb2d831c2587ced89c107d6802fef4ef30ec872cb551269df41fa"}
Jul 22 12:02:13.617976 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.617784    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pmljl" event={"ID":"464a2ed7-bcdb-4426-b488-9d826b166405","Type":"ContainerStarted","Data":"8a08608553c94ff2b502f0809464387c7a0b8bb20b2b5380d4d65ed2af6779a6"}
Jul 22 12:02:13.619989 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.619962    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s" event={"ID":"ee8865d5-4008-4275-9bd2-fb89c607064b","Type":"ContainerStarted","Data":"9f482d75b052c80eb21801b73a366133f581b115c3c22214bac6abddf26fec6f"}
Jul 22 12:02:13.620091 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.620074    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s" event={"ID":"ee8865d5-4008-4275-9bd2-fb89c607064b","Type":"ContainerStarted","Data":"727ee9f09df2933bb0e28d23dfe58d28f16abcea315c4a42fa06560596f8d699"}
Jul 22 12:02:13.620148 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.620135    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s" event={"ID":"ee8865d5-4008-4275-9bd2-fb89c607064b","Type":"ContainerStarted","Data":"afa4fa76d79d499c06a4d56fad543d07fd87d4adeed34b211b010a8da129d4e2"}
Jul 22 12:02:13.621346 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.621320    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7pdv9" event={"ID":"eef02bfb-180d-49b0-a4c6-52cf46557c5c","Type":"ContainerStarted","Data":"c944f38cdc203b939d49635682c7ab07412cf58e99861591043811cd1f7ba4ae"}
Jul 22 12:02:13.622941 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.622920    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr" event={"ID":"8115d8cc-5a84-43e3-8e8d-ce07ad68475d","Type":"ContainerStarted","Data":"6cae1a1d509352a7ddd9c4c4d7bc8c119cfe15d692a5559a0291a058ac901dcc"}
Jul 22 12:02:13.624269 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.624251    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6vjhd" event={"ID":"d39fb4a7-d50d-4ec1-8af1-a424fd0a453f","Type":"ContainerStarted","Data":"2a4ab2d7fa25b05d6e61dfab9030c58dd12000ef1fdbe65478a361cd4f29d333"}
Jul 22 12:02:13.625563 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.625541    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-shlxl" event={"ID":"1dfea4d2-cebc-4c94-b4ac-3641e198d0e9","Type":"ContainerStarted","Data":"bead47599070d87d040965c899e0bf7abd40f3a3d5f14c0a8876260280f531c8"}
Jul 22 12:02:13.633108 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.633066    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-107.ec2.internal" podStartSLOduration=21.633054182 podStartE2EDuration="21.633054182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:13.632668218 +0000 UTC m=+22.654689918" watchObservedRunningTime="2026-07-22 12:02:13.633054182 +0000 UTC m=+22.655075883"
Jul 22 12:02:13.651948 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.651896    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7pdv9" podStartSLOduration=2.196423717 podStartE2EDuration="21.651877421s" totalImagesPullingTime="19.455453704s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:13.651206565 +0000 UTC m=+22.673228293" watchObservedRunningTime="2026-07-22 12:02:13.651877421 +0000 UTC m=+22.673899124"
Jul 22 12:02:13.704212 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.704173    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-shlxl" podStartSLOduration=2.218864541 podStartE2EDuration="21.704157716s" totalImagesPullingTime="19.485293175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:13.7035241 +0000 UTC m=+22.725545800" watchObservedRunningTime="2026-07-22 12:02:13.704157716 +0000 UTC m=+22.726179417"
Jul 22 12:02:13.732102 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.732064    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6vjhd" podStartSLOduration=2.033419168 podStartE2EDuration="21.732051677s" totalImagesPullingTime="19.698632509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:13.731638164 +0000 UTC m=+22.753659864" watchObservedRunningTime="2026-07-22 12:02:13.732051677 +0000 UTC m=+22.754073378"
Jul 22 12:02:13.748933 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.748899    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pmljl" podStartSLOduration=2.248350664 podStartE2EDuration="21.748884248s" totalImagesPullingTime="19.500533584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:13.748295847 +0000 UTC m=+22.770317550" watchObservedRunningTime="2026-07-22 12:02:13.748884248 +0000 UTC m=+22.770905949"
Jul 22 12:02:13.774177 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:13.774137    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7l5c5" podStartSLOduration=2.278615204 podStartE2EDuration="21.774117776s" totalImagesPullingTime="19.495502572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:13.773525138 +0000 UTC m=+22.795546840" watchObservedRunningTime="2026-07-22 12:02:13.774117776 +0000 UTC m=+22.796139477"
Jul 22 12:02:14.631675 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:14.631638    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s" event={"ID":"ee8865d5-4008-4275-9bd2-fb89c607064b","Type":"ContainerStarted","Data":"7cc79ffdd458ea07b845241249892d412c48fa6ed2fc36c028fbd2b24d6d70d3"}
Jul 22 12:02:14.632210 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:14.631790    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s" event={"ID":"ee8865d5-4008-4275-9bd2-fb89c607064b","Type":"ContainerStarted","Data":"8685f4e7c96c44203fb84a65b46553d71939f7d17328d8d66ccecea24ca669fe"}
Jul 22 12:02:14.632210 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:14.631839    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s" event={"ID":"ee8865d5-4008-4275-9bd2-fb89c607064b","Type":"ContainerStarted","Data":"28c76d7dfd4a4976143cc5503b70cac552312b4ca44e1c9770a0566627bea5b9"}
Jul 22 12:02:15.519642 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:15.519603    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:15.519642 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:15.519633    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:15.519920 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:15.519600    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:15.519920 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:15.519736    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:15.519920 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:15.519856    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:15.520088 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:15.519940    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:15.635890 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:15.635846    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l6kj2" event={"ID":"864f6b9f-0aed-44a8-a12c-35f051e293e3","Type":"ContainerStarted","Data":"6964f43979e56fba805adc984ef7f978d8ae8cdc0bd768327ddfbb79ef85750b"}
Jul 22 12:02:15.662732 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:15.662689    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-l6kj2" podStartSLOduration=4.192702727 podStartE2EDuration="23.662674123s" totalImagesPullingTime="19.469971396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:15.662335831 +0000 UTC m=+24.684357533" watchObservedRunningTime="2026-07-22 12:02:15.662674123 +0000 UTC m=+24.684695824"
Jul 22 12:02:16.429450 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:16.429416    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:02:16.430260 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:16.430240    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:02:16.638709 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:16.638676    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:02:16.639194 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:16.639056    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pmljl"
Jul 22 12:02:17.519060 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:17.519038    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:17.519162 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:17.519058    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:17.519228 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:17.519168    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:17.519228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:17.519221    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:17.520588 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:17.519213    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:17.520588 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:17.519767    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:17.644834 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:17.644798    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s" event={"ID":"ee8865d5-4008-4275-9bd2-fb89c607064b","Type":"ContainerStarted","Data":"65a3563e51d3edb4efe693769af37c717dd62968db3953ca37252b25c31829d3"}
Jul 22 12:02:18.295731 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:18.295578    2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 12:02:18.444540 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:18.444449    2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T12:02:18.29572766Z","UUID":"4eb6714a-9d4c-4075-9a65-6226086dcd69","Handler":null,"Name":"","Endpoint":""}
Jul 22 12:02:18.445887 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:18.445867    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 12:02:18.446009 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:18.445896    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 12:02:18.648872 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:18.648841    2584 generic.go:356] "Generic (PLEG): container finished" podID="507d1473-a97c-4fe5-9829-4f7fbefc0967" containerID="a81367b2a287254b6a6dde19cfe7b552843efe7d51850de3c9292f4509a6538f" exitCode=0
Jul 22 12:02:18.649459 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:18.648910    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgbkp" event={"ID":"507d1473-a97c-4fe5-9829-4f7fbefc0967","Type":"ContainerDied","Data":"a81367b2a287254b6a6dde19cfe7b552843efe7d51850de3c9292f4509a6538f"}
Jul 22 12:02:18.650860 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:18.650837    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr" event={"ID":"8115d8cc-5a84-43e3-8e8d-ce07ad68475d","Type":"ContainerStarted","Data":"1cc728af7e381a0ea575dfd4091b3c0716fad520b572ee3dd32b100cca74dcd8"}
Jul 22 12:02:19.521655 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:19.521589    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:19.521787 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:19.521589    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:19.521787 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:19.521692    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:19.521787 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:19.521589    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:19.521904 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:19.521755    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:19.521904 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:19.521839    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:20.656481 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:20.656306    2584 generic.go:356] "Generic (PLEG): container finished" podID="507d1473-a97c-4fe5-9829-4f7fbefc0967" containerID="2b8d47bc7fac61cd3e776a5ed191c986bae2387cb2bd18588182e2853832f771" exitCode=0
Jul 22 12:02:20.657079 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:20.656389    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgbkp" event={"ID":"507d1473-a97c-4fe5-9829-4f7fbefc0967","Type":"ContainerDied","Data":"2b8d47bc7fac61cd3e776a5ed191c986bae2387cb2bd18588182e2853832f771"}
Jul 22 12:02:20.660236 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:20.660211    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s" event={"ID":"ee8865d5-4008-4275-9bd2-fb89c607064b","Type":"ContainerStarted","Data":"7b54fe2e455952f230adf04da7373f783d0797009d44b73e68cec3538874af59"}
Jul 22 12:02:20.660509 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:20.660486    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:02:20.660608 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:20.660563    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:02:20.660663 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:20.660613    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:02:20.675363 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:20.675344    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:02:20.675523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:20.675511    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:02:20.713479 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:20.713439    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s" podStartSLOduration=9.226035269 podStartE2EDuration="28.713426453s" totalImagesPullingTime="19.487391184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:20.713178767 +0000 UTC m=+29.735200486" watchObservedRunningTime="2026-07-22 12:02:20.713426453 +0000 UTC m=+29.735448165"
Jul 22 12:02:21.519648 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:21.519615    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:21.519808 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:21.519706    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:21.519808 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:21.519718    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:21.519808 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:21.519739    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:21.519965 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:21.519834    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:21.519965 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:21.519926    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:21.669026 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:21.668981    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr" event={"ID":"8115d8cc-5a84-43e3-8e8d-ce07ad68475d","Type":"ContainerStarted","Data":"b29c5d766f5b176d06c92da9f5037af2a54d06d03eb57694b1e02e0e521ca975"}
Jul 22 12:02:21.690894 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:21.690844    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkvxr" podStartSLOduration=3.144684505 podStartE2EDuration="29.690829224s" totalImagesPullingTime="26.546144719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:21.690391642 +0000 UTC m=+30.712413361" watchObservedRunningTime="2026-07-22 12:02:21.690829224 +0000 UTC m=+30.712850956"
Jul 22 12:02:22.041007 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:22.040972    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zkjd8"]
Jul 22 12:02:22.041154 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:22.041117    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:22.041289 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:22.041237    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:22.042925 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:22.042896    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kds22"]
Jul 22 12:02:22.043038 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:22.042990    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:22.043095 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:22.043070    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:22.055250 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:22.055186    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-48x6v"]
Jul 22 12:02:22.055350 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:22.055282    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:22.055422 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:22.055367    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:22.673636 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:22.673601    2584 generic.go:356] "Generic (PLEG): container finished" podID="507d1473-a97c-4fe5-9829-4f7fbefc0967" containerID="9160ab29411cff991fbe6ed319e267538dec1a8b204994997f61f6704eb0bdda" exitCode=0
Jul 22 12:02:22.674009 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:22.673686    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgbkp" event={"ID":"507d1473-a97c-4fe5-9829-4f7fbefc0967","Type":"ContainerDied","Data":"9160ab29411cff991fbe6ed319e267538dec1a8b204994997f61f6704eb0bdda"}
Jul 22 12:02:23.518908 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:23.518877    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:23.519070 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:23.518883    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:23.519070 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:23.518990    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:23.519196 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:23.519089    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:23.519196 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:23.519133    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:23.519196 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:23.519186    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:25.129482 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:25.129440    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:25.129942 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:25.129613    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:02:25.129942 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:25.129687    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs podName:34b5faa4-567b-421f-a050-87e201154841 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:57.129671374 +0000 UTC m=+66.151693058 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs") pod "network-metrics-daemon-zkjd8" (UID: "34b5faa4-567b-421f-a050-87e201154841") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:02:25.230782 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:25.230746    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:25.230966 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:25.230798    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47gmc\" (UniqueName: \"kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc\") pod \"network-check-target-48x6v\" (UID: \"f308d47a-3937-4b78-8dd1-199f13817de1\") " pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:25.230966 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:25.230853    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:02:25.230966 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:25.230898    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:02:25.230966 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:25.230916    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:02:25.230966 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:25.230921    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret podName:f78d8d30-61ed-4179-a3d4-bd7188b22aab nodeName:}" failed. No retries permitted until 2026-07-22 12:02:57.230902512 +0000 UTC m=+66.252924213 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret") pod "global-pull-secret-syncer-kds22" (UID: "f78d8d30-61ed-4179-a3d4-bd7188b22aab") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:02:25.230966 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:25.230928    2584 projected.go:196] Error preparing data for projected volume kube-api-access-47gmc for pod openshift-network-diagnostics/network-check-target-48x6v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:02:25.230966 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:25.230962    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc podName:f308d47a-3937-4b78-8dd1-199f13817de1 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:57.230951878 +0000 UTC m=+66.252973560 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-47gmc" (UniqueName: "kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc") pod "network-check-target-48x6v" (UID: "f308d47a-3937-4b78-8dd1-199f13817de1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:02:25.519757 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:25.519676    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:25.519757 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:25.519701    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:25.519965 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:25.519676    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:25.519965 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:25.519817    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkjd8" podUID="34b5faa4-567b-421f-a050-87e201154841"
Jul 22 12:02:25.519965 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:25.519935    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48x6v" podUID="f308d47a-3937-4b78-8dd1-199f13817de1"
Jul 22 12:02:25.520104 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:25.520019    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kds22" podUID="f78d8d30-61ed-4179-a3d4-bd7188b22aab"
Jul 22 12:02:26.188034 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.187740    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-107.ec2.internal" event="NodeReady"
Jul 22 12:02:26.188485 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.188094    2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 12:02:26.246767 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.246733    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lfqfn"]
Jul 22 12:02:26.248770 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.248752    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wjj7v"]
Jul 22 12:02:26.248890 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.248876    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.250588 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.250568    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wjj7v"
Jul 22 12:02:26.252050 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.252007    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 12:02:26.252187 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.252084    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q9wgb\""
Jul 22 12:02:26.252326 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.252309    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 12:02:26.253051 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.253031    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 12:02:26.253162 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.253144    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 12:02:26.253233 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.253218    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 12:02:26.253918 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.253897    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-82grb\""
Jul 22 12:02:26.261034 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.261013    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wjj7v"]
Jul 22 12:02:26.263726 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.263659    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lfqfn"]
Jul 22 12:02:26.337991 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.337963    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.337991 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.337996    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/262cb757-0f60-4d3b-a354-db74abbd3176-config-volume\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.338221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.338033    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5745c4de-e46c-4419-a288-70cf27b148a6-cert\") pod \"ingress-canary-wjj7v\" (UID: \"5745c4de-e46c-4419-a288-70cf27b148a6\") " pod="openshift-ingress-canary/ingress-canary-wjj7v"
Jul 22 12:02:26.338221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.338125    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f464w\" (UniqueName: \"kubernetes.io/projected/5745c4de-e46c-4419-a288-70cf27b148a6-kube-api-access-f464w\") pod \"ingress-canary-wjj7v\" (UID: \"5745c4de-e46c-4419-a288-70cf27b148a6\") " pod="openshift-ingress-canary/ingress-canary-wjj7v"
Jul 22 12:02:26.338221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.338170    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/262cb757-0f60-4d3b-a354-db74abbd3176-tmp-dir\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.338221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.338209    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww7gp\" (UniqueName: \"kubernetes.io/projected/262cb757-0f60-4d3b-a354-db74abbd3176-kube-api-access-ww7gp\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.439192 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.439102    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f464w\" (UniqueName: \"kubernetes.io/projected/5745c4de-e46c-4419-a288-70cf27b148a6-kube-api-access-f464w\") pod \"ingress-canary-wjj7v\" (UID: \"5745c4de-e46c-4419-a288-70cf27b148a6\") " pod="openshift-ingress-canary/ingress-canary-wjj7v"
Jul 22 12:02:26.439192 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.439160    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/262cb757-0f60-4d3b-a354-db74abbd3176-tmp-dir\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.439455 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.439316    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww7gp\" (UniqueName: \"kubernetes.io/projected/262cb757-0f60-4d3b-a354-db74abbd3176-kube-api-access-ww7gp\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.439455 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.439398    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.439455 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.439436    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/262cb757-0f60-4d3b-a354-db74abbd3176-config-volume\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.439593 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.439492    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5745c4de-e46c-4419-a288-70cf27b148a6-cert\") pod \"ingress-canary-wjj7v\" (UID: \"5745c4de-e46c-4419-a288-70cf27b148a6\") " pod="openshift-ingress-canary/ingress-canary-wjj7v"
Jul 22 12:02:26.439593 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:26.439503    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:02:26.439593 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.439512    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/262cb757-0f60-4d3b-a354-db74abbd3176-tmp-dir\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.439593 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:26.439558    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls podName:262cb757-0f60-4d3b-a354-db74abbd3176 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:26.939540464 +0000 UTC m=+35.961562157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls") pod "dns-default-lfqfn" (UID: "262cb757-0f60-4d3b-a354-db74abbd3176") : secret "dns-default-metrics-tls" not found
Jul 22 12:02:26.440015 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.439996    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/262cb757-0f60-4d3b-a354-db74abbd3176-config-volume\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.444204 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.444185    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5745c4de-e46c-4419-a288-70cf27b148a6-cert\") pod \"ingress-canary-wjj7v\" (UID: \"5745c4de-e46c-4419-a288-70cf27b148a6\") " pod="openshift-ingress-canary/ingress-canary-wjj7v"
Jul 22 12:02:26.448950 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.448907    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww7gp\" (UniqueName: \"kubernetes.io/projected/262cb757-0f60-4d3b-a354-db74abbd3176-kube-api-access-ww7gp\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.449967 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.449942    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f464w\" (UniqueName: \"kubernetes.io/projected/5745c4de-e46c-4419-a288-70cf27b148a6-kube-api-access-f464w\") pod \"ingress-canary-wjj7v\" (UID: \"5745c4de-e46c-4419-a288-70cf27b148a6\") " pod="openshift-ingress-canary/ingress-canary-wjj7v"
Jul 22 12:02:26.569475 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.569444    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wjj7v"
Jul 22 12:02:26.712361 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.712285    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wjj7v"]
Jul 22 12:02:26.715698 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:02:26.715668    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5745c4de_e46c_4419_a288_70cf27b148a6.slice/crio-1f7e84a3b5f38c5ee9cb4910ba3625bb0e257057a0b1d690f1f70e6492e55e6d WatchSource:0}: Error finding container 1f7e84a3b5f38c5ee9cb4910ba3625bb0e257057a0b1d690f1f70e6492e55e6d: Status 404 returned error can't find the container with id 1f7e84a3b5f38c5ee9cb4910ba3625bb0e257057a0b1d690f1f70e6492e55e6d
Jul 22 12:02:26.942604 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:26.942571    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:26.942776 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:26.942738    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:02:26.942832 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:26.942817    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls podName:262cb757-0f60-4d3b-a354-db74abbd3176 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:27.942799101 +0000 UTC m=+36.964820788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls") pod "dns-default-lfqfn" (UID: "262cb757-0f60-4d3b-a354-db74abbd3176") : secret "dns-default-metrics-tls" not found
Jul 22 12:02:27.519576 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:27.519542    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:27.520414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:27.519542    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:27.520414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:27.519545    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:27.524141 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:27.524116    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 12:02:27.524141 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:27.524136    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 12:02:27.524315 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:27.524124    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4n945\""
Jul 22 12:02:27.524315 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:27.524197    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 12:02:27.524315 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:27.524275    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-856ws\""
Jul 22 12:02:27.524493 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:27.524454    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 12:02:27.687749 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:27.687708    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wjj7v" event={"ID":"5745c4de-e46c-4419-a288-70cf27b148a6","Type":"ContainerStarted","Data":"1f7e84a3b5f38c5ee9cb4910ba3625bb0e257057a0b1d690f1f70e6492e55e6d"}
Jul 22 12:02:27.950836 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:27.950802    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:27.951012 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:27.950974    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:02:27.951064 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:27.951053    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls podName:262cb757-0f60-4d3b-a354-db74abbd3176 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:29.951031011 +0000 UTC m=+38.973052698 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls") pod "dns-default-lfqfn" (UID: "262cb757-0f60-4d3b-a354-db74abbd3176") : secret "dns-default-metrics-tls" not found
Jul 22 12:02:28.694174 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:28.693885    2584 generic.go:356] "Generic (PLEG): container finished" podID="507d1473-a97c-4fe5-9829-4f7fbefc0967" containerID="452495f06dca8f824f29ee3bc59a5c4f11fcc94ae885628bac27792898a4fc00" exitCode=0
Jul 22 12:02:28.694174 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:28.693976    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgbkp" event={"ID":"507d1473-a97c-4fe5-9829-4f7fbefc0967","Type":"ContainerDied","Data":"452495f06dca8f824f29ee3bc59a5c4f11fcc94ae885628bac27792898a4fc00"}
Jul 22 12:02:29.700499 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:29.700471    2584 generic.go:356] "Generic (PLEG): container finished" podID="507d1473-a97c-4fe5-9829-4f7fbefc0967" containerID="1858174a37f6e3aa0d49a8c7310f2379ec6f062b77ebdeda36f87bd6ff901e0e" exitCode=0
Jul 22 12:02:29.700945 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:29.700530    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgbkp" event={"ID":"507d1473-a97c-4fe5-9829-4f7fbefc0967","Type":"ContainerDied","Data":"1858174a37f6e3aa0d49a8c7310f2379ec6f062b77ebdeda36f87bd6ff901e0e"}
Jul 22 12:02:29.967640 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:29.967556    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:29.967796 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:29.967677    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:02:29.967796 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:29.967760    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls podName:262cb757-0f60-4d3b-a354-db74abbd3176 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:33.967740611 +0000 UTC m=+42.989762312 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls") pod "dns-default-lfqfn" (UID: "262cb757-0f60-4d3b-a354-db74abbd3176") : secret "dns-default-metrics-tls" not found
Jul 22 12:02:30.706316 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:30.706236    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jgbkp" event={"ID":"507d1473-a97c-4fe5-9829-4f7fbefc0967","Type":"ContainerStarted","Data":"8adf5d7bc5b3a223b89362cd5904f7ce9ff00d50b008f129b2a26f048eb6d4bb"}
Jul 22 12:02:30.707671 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:30.707649    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wjj7v" event={"ID":"5745c4de-e46c-4419-a288-70cf27b148a6","Type":"ContainerStarted","Data":"fde4aac842e10794f7efa22c1ac02355512d701e693e5ba181d92dd1acc6e034"}
Jul 22 12:02:30.736458 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:30.736418    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jgbkp" podStartSLOduration=7.426875395 podStartE2EDuration="38.736405456s" totalImagesPullingTime="31.309530061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:30.734888955 +0000 UTC m=+39.756910655" watchObservedRunningTime="2026-07-22 12:02:30.736405456 +0000 UTC m=+39.758427157"
Jul 22 12:02:30.754716 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:30.754675    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wjj7v" podStartSLOduration=1.118399779 podStartE2EDuration="4.754662962s" totalImagesPullingTime="3.636263183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:30.754126155 +0000 UTC m=+39.776147878" watchObservedRunningTime="2026-07-22 12:02:30.754662962 +0000 UTC m=+39.776684663"
Jul 22 12:02:33.997249 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:33.997200    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:33.997659 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:33.997331    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:02:33.997659 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:33.997402    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls podName:262cb757-0f60-4d3b-a354-db74abbd3176 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:41.997371303 +0000 UTC m=+51.019392987 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls") pod "dns-default-lfqfn" (UID: "262cb757-0f60-4d3b-a354-db74abbd3176") : secret "dns-default-metrics-tls" not found
Jul 22 12:02:42.051888 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:42.051845    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:42.052345 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:42.051996    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:02:42.052345 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:42.052061    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls podName:262cb757-0f60-4d3b-a354-db74abbd3176 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:58.0520455 +0000 UTC m=+67.074067182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls") pod "dns-default-lfqfn" (UID: "262cb757-0f60-4d3b-a354-db74abbd3176") : secret "dns-default-metrics-tls" not found
Jul 22 12:02:52.692294 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:52.692266    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hcr2s"
Jul 22 12:02:57.151955 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.151907    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:02:57.154932 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.154916    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 12:02:57.163110 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:57.163090    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 12:02:57.163178 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:57.163167    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs podName:34b5faa4-567b-421f-a050-87e201154841 nodeName:}" failed. No retries permitted until 2026-07-22 12:04:01.163146131 +0000 UTC m=+130.185167812 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs") pod "network-metrics-daemon-zkjd8" (UID: "34b5faa4-567b-421f-a050-87e201154841") : secret "metrics-daemon-secret" not found
Jul 22 12:02:57.253208 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.253186    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47gmc\" (UniqueName: \"kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc\") pod \"network-check-target-48x6v\" (UID: \"f308d47a-3937-4b78-8dd1-199f13817de1\") " pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:57.253275 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.253259    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:57.256634 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.256605    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 12:02:57.256634 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.256628    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 12:02:57.266534 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.266515    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 12:02:57.267210 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.267190    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f78d8d30-61ed-4179-a3d4-bd7188b22aab-original-pull-secret\") pod \"global-pull-secret-syncer-kds22\" (UID: \"f78d8d30-61ed-4179-a3d4-bd7188b22aab\") " pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:57.276615 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.276594    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47gmc\" (UniqueName: \"kubernetes.io/projected/f308d47a-3937-4b78-8dd1-199f13817de1-kube-api-access-47gmc\") pod \"network-check-target-48x6v\" (UID: \"f308d47a-3937-4b78-8dd1-199f13817de1\") " pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:57.534787 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.534713    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-856ws\""
Jul 22 12:02:57.537946 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.537930    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kds22"
Jul 22 12:02:57.542607 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.542592    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:02:57.668194 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.668164    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kds22"]
Jul 22 12:02:57.670987 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:02:57.670962    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf78d8d30_61ed_4179_a3d4_bd7188b22aab.slice/crio-b83c8020c3e1bd2d03ba9767fdba7116e8f9ca3876ff1b12788a7595c73b61f2 WatchSource:0}: Error finding container b83c8020c3e1bd2d03ba9767fdba7116e8f9ca3876ff1b12788a7595c73b61f2: Status 404 returned error can't find the container with id b83c8020c3e1bd2d03ba9767fdba7116e8f9ca3876ff1b12788a7595c73b61f2
Jul 22 12:02:57.684524 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.684486    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-48x6v"]
Jul 22 12:02:57.687850 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:02:57.687824    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf308d47a_3937_4b78_8dd1_199f13817de1.slice/crio-a750b95e391af5e6547c14c363b5797cb5c90c0570d7162163858acd5a1e2edd WatchSource:0}: Error finding container a750b95e391af5e6547c14c363b5797cb5c90c0570d7162163858acd5a1e2edd: Status 404 returned error can't find the container with id a750b95e391af5e6547c14c363b5797cb5c90c0570d7162163858acd5a1e2edd
Jul 22 12:02:57.770451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.770418    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kds22" event={"ID":"f78d8d30-61ed-4179-a3d4-bd7188b22aab","Type":"ContainerStarted","Data":"b83c8020c3e1bd2d03ba9767fdba7116e8f9ca3876ff1b12788a7595c73b61f2"}
Jul 22 12:02:57.771551 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:57.771527    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-48x6v" event={"ID":"f308d47a-3937-4b78-8dd1-199f13817de1","Type":"ContainerStarted","Data":"a750b95e391af5e6547c14c363b5797cb5c90c0570d7162163858acd5a1e2edd"}
Jul 22 12:02:58.056955 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:02:58.056921    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:02:58.057098 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:58.057042    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:02:58.057155 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:02:58.057104    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls podName:262cb757-0f60-4d3b-a354-db74abbd3176 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:30.057091113 +0000 UTC m=+99.079112792 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls") pod "dns-default-lfqfn" (UID: "262cb757-0f60-4d3b-a354-db74abbd3176") : secret "dns-default-metrics-tls" not found
Jul 22 12:03:02.485304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.485278    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"]
Jul 22 12:03:02.487143 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.487128    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:02.490031 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.490011    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 12:03:02.490173 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.490091    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 12:03:02.490317 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.490300    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 22 12:03:02.491290 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.491276    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kflpx\""
Jul 22 12:03:02.493173 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.493153    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 22 12:03:02.506304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.506284    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"]
Jul 22 12:03:02.583228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.583197    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64d58b7fcb-9b8kf"]
Jul 22 12:03:02.585017 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.585002    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.590222 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.590192    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 22 12:03:02.590364 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.590243    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 22 12:03:02.590364 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.590248    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 12:03:02.590364 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.590197    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 22 12:03:02.590533 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.590368    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2trhd\""
Jul 22 12:03:02.590533 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.590197    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 22 12:03:02.590533 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.590203    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 12:03:02.591080 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.591060    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/59d9f112-d3ab-47da-8873-d13561388b67-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:02.591207 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.591096    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prcm6\" (UniqueName: \"kubernetes.io/projected/59d9f112-d3ab-47da-8873-d13561388b67-kube-api-access-prcm6\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:02.591207 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.591189    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:02.600445 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.600424    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64d58b7fcb-9b8kf"]
Jul 22 12:03:02.692223 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.692192    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:02.692507 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.692240    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.692507 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.692277    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-default-certificate\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.692507 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.692306    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/59d9f112-d3ab-47da-8873-d13561388b67-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:02.692507 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.692328    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-stats-auth\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.692507 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:02.692331    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:02.692507 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.692396    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prcm6\" (UniqueName: \"kubernetes.io/projected/59d9f112-d3ab-47da-8873-d13561388b67-kube-api-access-prcm6\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:02.692507 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:02.692430    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls podName:59d9f112-d3ab-47da-8873-d13561388b67 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:03.192409701 +0000 UTC m=+72.214431399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cxxpr" (UID: "59d9f112-d3ab-47da-8873-d13561388b67") : secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:02.692507 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.692470    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqhn5\" (UniqueName: \"kubernetes.io/projected/62903732-d37c-4b43-b094-963c0910ac8b-kube-api-access-fqhn5\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.692507 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.692503    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.692919 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.692903    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/59d9f112-d3ab-47da-8873-d13561388b67-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:02.701837 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.701812    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prcm6\" (UniqueName: \"kubernetes.io/projected/59d9f112-d3ab-47da-8873-d13561388b67-kube-api-access-prcm6\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:02.786301 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.786221    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-48x6v" event={"ID":"f308d47a-3937-4b78-8dd1-199f13817de1","Type":"ContainerStarted","Data":"6215ea531b966c7a7618d7ffe8e3b6121cb9679fcf79159af55547ca3e497686"}
Jul 22 12:03:02.786301 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.786290    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:03:02.787646 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.787625    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kds22" event={"ID":"f78d8d30-61ed-4179-a3d4-bd7188b22aab","Type":"ContainerStarted","Data":"e665bf5908121abfe5c10caa4d8f3c8aad08a976c6ae6b85642635bfb32aec11"}
Jul 22 12:03:02.793669 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.793651    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-default-certificate\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.793720 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.793680    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-stats-auth\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.793720 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.793697    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqhn5\" (UniqueName: \"kubernetes.io/projected/62903732-d37c-4b43-b094-963c0910ac8b-kube-api-access-fqhn5\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.793836 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.793817    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.793925 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.793912    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.794018 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:02.793957    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 12:03:02.794073 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:02.794042    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:03.294021224 +0000 UTC m=+72.316042917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : secret "router-metrics-certs-default" not found
Jul 22 12:03:02.794073 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:02.794064    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:03.294053395 +0000 UTC m=+72.316075079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : configmap references non-existent config key: service-ca.crt
Jul 22 12:03:02.795913 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.795887    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-stats-auth\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.795994 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.795912    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-default-certificate\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.805129 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.805109    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqhn5\" (UniqueName: \"kubernetes.io/projected/62903732-d37c-4b43-b094-963c0910ac8b-kube-api-access-fqhn5\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:02.805220 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.805188    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-48x6v" podStartSLOduration=66.396901112 podStartE2EDuration="1m10.805175481s" totalImagesPullingTime="4.408274369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:02.804668622 +0000 UTC m=+71.826690323" watchObservedRunningTime="2026-07-22 12:03:02.805175481 +0000 UTC m=+71.827197160"
Jul 22 12:03:02.823652 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:02.823609    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kds22" podStartSLOduration=66.39593306 podStartE2EDuration="1m10.823597073s" totalImagesPullingTime="4.427664013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:02.823283553 +0000 UTC m=+71.845305273" watchObservedRunningTime="2026-07-22 12:03:02.823597073 +0000 UTC m=+71.845618796"
Jul 22 12:03:03.197027 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:03.196984    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:03.197193 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:03.197121    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:03.197193 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:03.197184    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls podName:59d9f112-d3ab-47da-8873-d13561388b67 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:04.197169712 +0000 UTC m=+73.219191392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cxxpr" (UID: "59d9f112-d3ab-47da-8873-d13561388b67") : secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:03.297639 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:03.297597    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:03.297738 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:03.297660    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:03.297795 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:03.297778    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:04.297761917 +0000 UTC m=+73.319783597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : configmap references non-existent config key: service-ca.crt
Jul 22 12:03:03.297839 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:03.297779    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 12:03:03.297839 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:03.297818    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:04.297810377 +0000 UTC m=+73.319832056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : secret "router-metrics-certs-default" not found
Jul 22 12:03:04.203618 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:04.203569    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:04.203994 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:04.203714    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:04.203994 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:04.203781    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls podName:59d9f112-d3ab-47da-8873-d13561388b67 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:06.20376248 +0000 UTC m=+75.225784159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cxxpr" (UID: "59d9f112-d3ab-47da-8873-d13561388b67") : secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:04.304155 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:04.304109    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:04.304155 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:04.304166    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:04.304368 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:04.304265    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:06.3042493 +0000 UTC m=+75.326270979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : configmap references non-existent config key: service-ca.crt
Jul 22 12:03:04.304368 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:04.304268    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 12:03:04.304368 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:04.304307    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:06.304299513 +0000 UTC m=+75.326321243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : secret "router-metrics-certs-default" not found
Jul 22 12:03:06.217937 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.217898    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:06.218304 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:06.218040    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:06.218304 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:06.218102    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls podName:59d9f112-d3ab-47da-8873-d13561388b67 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:10.21808524 +0000 UTC m=+79.240106940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cxxpr" (UID: "59d9f112-d3ab-47da-8873-d13561388b67") : secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:06.318605 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.318564    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:06.318741 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.318652    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:06.318741 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:06.318725    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 12:03:06.318877 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:06.318770    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:10.318756828 +0000 UTC m=+79.340778506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : configmap references non-existent config key: service-ca.crt
Jul 22 12:03:06.318877 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:06.318784    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:10.31877854 +0000 UTC m=+79.340800219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : secret "router-metrics-certs-default" not found
Jul 22 12:03:06.773750 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.773716    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hq468"]
Jul 22 12:03:06.776808 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.776789    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.783166 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.782969    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 12:03:06.783166 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.783056    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tmstd\""
Jul 22 12:03:06.785829 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.785811    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 12:03:06.785952 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.785880    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 12:03:06.785952 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.785940    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 12:03:06.811527 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.811506    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hq468"]
Jul 22 12:03:06.822747 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.822721    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/01d69365-7cc5-44c0-b707-0f5c2d343d73-crio-socket\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.822847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.822755    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.822847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.822782    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mxwq\" (UniqueName: \"kubernetes.io/projected/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-api-access-8mxwq\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.822923 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.822867    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.822957 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.822943    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/01d69365-7cc5-44c0-b707-0f5c2d343d73-data-volume\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.923393 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.923348    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/01d69365-7cc5-44c0-b707-0f5c2d343d73-data-volume\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.923566 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.923416    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/01d69365-7cc5-44c0-b707-0f5c2d343d73-crio-socket\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.923566 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.923436    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.923566 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.923452    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mxwq\" (UniqueName: \"kubernetes.io/projected/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-api-access-8mxwq\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.923566 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.923485    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.923566 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.923488    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/01d69365-7cc5-44c0-b707-0f5c2d343d73-crio-socket\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.923566 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:06.923562    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:03:06.923836 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:06.923603    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:06.923836 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:06.923625    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm podName:01d69365-7cc5-44c0-b707-0f5c2d343d73 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:07.423607448 +0000 UTC m=+76.445629129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hq468" (UID: "01d69365-7cc5-44c0-b707-0f5c2d343d73") : configmap "kube-rbac-proxy" not found
Jul 22 12:03:06.923836 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:06.923655    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls podName:01d69365-7cc5-44c0-b707-0f5c2d343d73 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:07.423643159 +0000 UTC m=+76.445664842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hq468" (UID: "01d69365-7cc5-44c0-b707-0f5c2d343d73") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:06.923836 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.923745    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/01d69365-7cc5-44c0-b707-0f5c2d343d73-data-volume\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:06.936169 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:06.936141    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mxwq\" (UniqueName: \"kubernetes.io/projected/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-api-access-8mxwq\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:07.427910 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:07.427876    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:07.428281 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:07.427927    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:07.428281 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:07.428009    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:03:07.428281 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:07.428050    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:07.428281 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:07.428086    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm podName:01d69365-7cc5-44c0-b707-0f5c2d343d73 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:08.428069013 +0000 UTC m=+77.450090711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hq468" (UID: "01d69365-7cc5-44c0-b707-0f5c2d343d73") : configmap "kube-rbac-proxy" not found
Jul 22 12:03:07.428281 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:07.428123    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls podName:01d69365-7cc5-44c0-b707-0f5c2d343d73 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:08.428111528 +0000 UTC m=+77.450133207 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hq468" (UID: "01d69365-7cc5-44c0-b707-0f5c2d343d73") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:08.435633 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:08.435588    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:08.436040 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:08.435655    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:08.436040 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:08.435717    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:03:08.436040 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:08.435778    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:08.436040 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:08.435791    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm podName:01d69365-7cc5-44c0-b707-0f5c2d343d73 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:10.43577657 +0000 UTC m=+79.457798249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hq468" (UID: "01d69365-7cc5-44c0-b707-0f5c2d343d73") : configmap "kube-rbac-proxy" not found
Jul 22 12:03:08.436040 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:08.435819    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls podName:01d69365-7cc5-44c0-b707-0f5c2d343d73 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:10.435806183 +0000 UTC m=+79.457827861 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hq468" (UID: "01d69365-7cc5-44c0-b707-0f5c2d343d73") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:08.566217 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:08.566188    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-shlxl_1dfea4d2-cebc-4c94-b4ac-3641e198d0e9/dns-node-resolver/0.log"
Jul 22 12:03:09.564855 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:09.564832    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7pdv9_eef02bfb-180d-49b0-a4c6-52cf46557c5c/node-ca/0.log"
Jul 22 12:03:10.249039 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:10.249005    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:10.249224 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:10.249173    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:10.249292 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:10.249252    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls podName:59d9f112-d3ab-47da-8873-d13561388b67 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:18.249231591 +0000 UTC m=+87.271253287 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cxxpr" (UID: "59d9f112-d3ab-47da-8873-d13561388b67") : secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:10.350176 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:10.350145    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:10.350355 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:10.350192    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:10.350355 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:10.350281    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 12:03:10.350355 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:10.350323    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:18.350304242 +0000 UTC m=+87.372325920 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : configmap references non-existent config key: service-ca.crt
Jul 22 12:03:10.350557 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:10.350402    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:18.350370366 +0000 UTC m=+87.372392051 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : secret "router-metrics-certs-default" not found
Jul 22 12:03:10.451531 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:10.451506    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:10.451668 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:10.451549    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:10.451668 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:10.451626    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:03:10.451668 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:10.451653    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:10.451782 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:10.451688    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm podName:01d69365-7cc5-44c0-b707-0f5c2d343d73 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:14.451671616 +0000 UTC m=+83.473693308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hq468" (UID: "01d69365-7cc5-44c0-b707-0f5c2d343d73") : configmap "kube-rbac-proxy" not found
Jul 22 12:03:10.451782 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:10.451703    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls podName:01d69365-7cc5-44c0-b707-0f5c2d343d73 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:14.451697068 +0000 UTC m=+83.473718746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hq468" (UID: "01d69365-7cc5-44c0-b707-0f5c2d343d73") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:10.564816 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:10.564748    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wjj7v_5745c4de-e46c-4419-a288-70cf27b148a6/serve-healthcheck-canary/0.log"
Jul 22 12:03:12.587148 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.587112    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"]
Jul 22 12:03:12.589054 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.589040    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"
Jul 22 12:03:12.591493 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.591469    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 22 12:03:12.591664 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.591645    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 22 12:03:12.591738 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.591722    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 22 12:03:12.591738 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.591734    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 22 12:03:12.592733 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.592716    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-928db\""
Jul 22 12:03:12.602897 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.602878    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"]
Jul 22 12:03:12.668464 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.668439    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/000c11b2-8abd-49ff-a384-860d4239e130-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5k8vw\" (UID: \"000c11b2-8abd-49ff-a384-860d4239e130\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"
Jul 22 12:03:12.668464 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.668464    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/000c11b2-8abd-49ff-a384-860d4239e130-config\") pod \"service-ca-operator-5bcbc94c78-5k8vw\" (UID: \"000c11b2-8abd-49ff-a384-860d4239e130\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"
Jul 22 12:03:12.668592 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.668541    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkpkh\" (UniqueName: \"kubernetes.io/projected/000c11b2-8abd-49ff-a384-860d4239e130-kube-api-access-kkpkh\") pod \"service-ca-operator-5bcbc94c78-5k8vw\" (UID: \"000c11b2-8abd-49ff-a384-860d4239e130\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"
Jul 22 12:03:12.769015 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.768924    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkpkh\" (UniqueName: \"kubernetes.io/projected/000c11b2-8abd-49ff-a384-860d4239e130-kube-api-access-kkpkh\") pod \"service-ca-operator-5bcbc94c78-5k8vw\" (UID: \"000c11b2-8abd-49ff-a384-860d4239e130\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"
Jul 22 12:03:12.769118 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.769014    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/000c11b2-8abd-49ff-a384-860d4239e130-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5k8vw\" (UID: \"000c11b2-8abd-49ff-a384-860d4239e130\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"
Jul 22 12:03:12.769118 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.769041    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/000c11b2-8abd-49ff-a384-860d4239e130-config\") pod \"service-ca-operator-5bcbc94c78-5k8vw\" (UID: \"000c11b2-8abd-49ff-a384-860d4239e130\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"
Jul 22 12:03:12.769615 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.769598    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/000c11b2-8abd-49ff-a384-860d4239e130-config\") pod \"service-ca-operator-5bcbc94c78-5k8vw\" (UID: \"000c11b2-8abd-49ff-a384-860d4239e130\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"
Jul 22 12:03:12.771136 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.771119    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/000c11b2-8abd-49ff-a384-860d4239e130-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5k8vw\" (UID: \"000c11b2-8abd-49ff-a384-860d4239e130\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"
Jul 22 12:03:12.777978 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.777950    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkpkh\" (UniqueName: \"kubernetes.io/projected/000c11b2-8abd-49ff-a384-860d4239e130-kube-api-access-kkpkh\") pod \"service-ca-operator-5bcbc94c78-5k8vw\" (UID: \"000c11b2-8abd-49ff-a384-860d4239e130\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"
Jul 22 12:03:12.897574 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:12.897548    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"
Jul 22 12:03:13.013680 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:13.013649    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw"]
Jul 22 12:03:13.016987 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:13.016956    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000c11b2_8abd_49ff_a384_860d4239e130.slice/crio-cf7ba9d6c97702ad50db48e5e546905eb42e3ace2c195a13b8251bf46c8110ed WatchSource:0}: Error finding container cf7ba9d6c97702ad50db48e5e546905eb42e3ace2c195a13b8251bf46c8110ed: Status 404 returned error can't find the container with id cf7ba9d6c97702ad50db48e5e546905eb42e3ace2c195a13b8251bf46c8110ed
Jul 22 12:03:13.815207 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:13.815168    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw" event={"ID":"000c11b2-8abd-49ff-a384-860d4239e130","Type":"ContainerStarted","Data":"cf7ba9d6c97702ad50db48e5e546905eb42e3ace2c195a13b8251bf46c8110ed"}
Jul 22 12:03:14.481466 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:14.481435    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:14.481642 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:14.481500    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:14.481642 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:14.481560    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:03:14.481642 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:14.481620    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:14.481783 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:14.481681    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls podName:01d69365-7cc5-44c0-b707-0f5c2d343d73 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:22.481661054 +0000 UTC m=+91.503682760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hq468" (UID: "01d69365-7cc5-44c0-b707-0f5c2d343d73") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:14.481783 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:14.481735    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm podName:01d69365-7cc5-44c0-b707-0f5c2d343d73 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:22.481723506 +0000 UTC m=+91.503745190 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hq468" (UID: "01d69365-7cc5-44c0-b707-0f5c2d343d73") : configmap "kube-rbac-proxy" not found
Jul 22 12:03:15.821239 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:15.821199    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw" event={"ID":"000c11b2-8abd-49ff-a384-860d4239e130","Type":"ContainerStarted","Data":"51fb9e5ad1f1548046a20a57e60365544136f6857edee293e5ce8bcb534ed506"}
Jul 22 12:03:15.843988 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:15.843934    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw" podStartSLOduration=1.87112546 podStartE2EDuration="3.843913453s" totalImagesPullingTime="1.972787993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:15.842777659 +0000 UTC m=+84.864799360" watchObservedRunningTime="2026-07-22 12:03:15.843913453 +0000 UTC m=+84.865935155"
Jul 22 12:03:18.298965 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.298930    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn"]
Jul 22 12:03:18.301054 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.301038    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn"
Jul 22 12:03:18.303471 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.303451    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 22 12:03:18.303538 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.303470    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 22 12:03:18.304577 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.304561    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mb2wp\""
Jul 22 12:03:18.310358 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.310337    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn"]
Jul 22 12:03:18.312491 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.312468    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:18.312650 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:18.312633    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:18.312730 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:18.312718    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls podName:59d9f112-d3ab-47da-8873-d13561388b67 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:34.312698271 +0000 UTC m=+103.334719964 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cxxpr" (UID: "59d9f112-d3ab-47da-8873-d13561388b67") : secret "cluster-monitoring-operator-tls" not found
Jul 22 12:03:18.413843 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.413811    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:18.414059 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.413859    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:18.414059 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.413884    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh9bw\" (UniqueName: \"kubernetes.io/projected/b2df455b-68ea-4945-b5ad-13ea85c7f983-kube-api-access-xh9bw\") pod \"migrator-7cc9c7ffd7-xqcbn\" (UID: \"b2df455b-68ea-4945-b5ad-13ea85c7f983\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn"
Jul 22 12:03:18.414059 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:18.413977    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:34.413951869 +0000 UTC m=+103.435973548 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : configmap references non-existent config key: service-ca.crt
Jul 22 12:03:18.414059 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:18.413987    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 12:03:18.414059 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:18.414056    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs podName:62903732-d37c-4b43-b094-963c0910ac8b nodeName:}" failed. No retries permitted until 2026-07-22 12:03:34.414038596 +0000 UTC m=+103.436060289 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs") pod "router-default-64d58b7fcb-9b8kf" (UID: "62903732-d37c-4b43-b094-963c0910ac8b") : secret "router-metrics-certs-default" not found
Jul 22 12:03:18.514619 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.514590    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh9bw\" (UniqueName: \"kubernetes.io/projected/b2df455b-68ea-4945-b5ad-13ea85c7f983-kube-api-access-xh9bw\") pod \"migrator-7cc9c7ffd7-xqcbn\" (UID: \"b2df455b-68ea-4945-b5ad-13ea85c7f983\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn"
Jul 22 12:03:18.523739 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.523715    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh9bw\" (UniqueName: \"kubernetes.io/projected/b2df455b-68ea-4945-b5ad-13ea85c7f983-kube-api-access-xh9bw\") pod \"migrator-7cc9c7ffd7-xqcbn\" (UID: \"b2df455b-68ea-4945-b5ad-13ea85c7f983\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn"
Jul 22 12:03:18.609854 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.609834    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn"
Jul 22 12:03:18.729142 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.729126    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn"]
Jul 22 12:03:18.731501 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:18.731471    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2df455b_68ea_4945_b5ad_13ea85c7f983.slice/crio-04dd4eb8180e4b1d9083d9e02fe539aadffdbbc8b7583d52176bef0fc4305172 WatchSource:0}: Error finding container 04dd4eb8180e4b1d9083d9e02fe539aadffdbbc8b7583d52176bef0fc4305172: Status 404 returned error can't find the container with id 04dd4eb8180e4b1d9083d9e02fe539aadffdbbc8b7583d52176bef0fc4305172
Jul 22 12:03:18.829089 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:18.829059    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn" event={"ID":"b2df455b-68ea-4945-b5ad-13ea85c7f983","Type":"ContainerStarted","Data":"04dd4eb8180e4b1d9083d9e02fe539aadffdbbc8b7583d52176bef0fc4305172"}
Jul 22 12:03:21.838355 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:21.838316    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn" event={"ID":"b2df455b-68ea-4945-b5ad-13ea85c7f983","Type":"ContainerStarted","Data":"aff88d0f787fc84ceaa132567470b6ed310fe9e795e8deb76a68a0114a76e77d"}
Jul 22 12:03:21.838740 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:21.838372    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn" event={"ID":"b2df455b-68ea-4945-b5ad-13ea85c7f983","Type":"ContainerStarted","Data":"38174db57f1f450017db4982e9d052dde2f3a1e3cc2785955e0c8da98c6ad78d"}
Jul 22 12:03:21.859444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:21.859397    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-xqcbn" podStartSLOduration=1.7792363089999998 podStartE2EDuration="3.85935968s" totalImagesPullingTime="2.080123371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:21.858827576 +0000 UTC m=+90.880849277" watchObservedRunningTime="2026-07-22 12:03:21.85935968 +0000 UTC m=+90.881381381"
Jul 22 12:03:22.544215 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:22.544183    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:22.544410 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:22.544235    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:22.544410 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:22.544335    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:22.544495 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:22.544434    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls podName:01d69365-7cc5-44c0-b707-0f5c2d343d73 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:38.544373685 +0000 UTC m=+107.566395364 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hq468" (UID: "01d69365-7cc5-44c0-b707-0f5c2d343d73") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:22.544865 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:22.544846    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/01d69365-7cc5-44c0-b707-0f5c2d343d73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:30.104337 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:30.104301    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:03:30.106549 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:30.106524    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/262cb757-0f60-4d3b-a354-db74abbd3176-metrics-tls\") pod \"dns-default-lfqfn\" (UID: \"262cb757-0f60-4d3b-a354-db74abbd3176\") " pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:03:30.165587 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:30.165562    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q9wgb\""
Jul 22 12:03:30.173474 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:30.173447    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:03:30.296034 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:30.296004    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lfqfn"]
Jul 22 12:03:30.299248 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:30.299222    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod262cb757_0f60_4d3b_a354_db74abbd3176.slice/crio-d8d889853224a0c6299e5171247c7992d73b3ffc99144546c80ebe13cb6dce22 WatchSource:0}: Error finding container d8d889853224a0c6299e5171247c7992d73b3ffc99144546c80ebe13cb6dce22: Status 404 returned error can't find the container with id d8d889853224a0c6299e5171247c7992d73b3ffc99144546c80ebe13cb6dce22
Jul 22 12:03:30.864737 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:30.864703    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lfqfn" event={"ID":"262cb757-0f60-4d3b-a354-db74abbd3176","Type":"ContainerStarted","Data":"d8d889853224a0c6299e5171247c7992d73b3ffc99144546c80ebe13cb6dce22"}
Jul 22 12:03:32.873116 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:32.873076    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lfqfn" event={"ID":"262cb757-0f60-4d3b-a354-db74abbd3176","Type":"ContainerStarted","Data":"762a6976566c5e2fac99c59d47227a99ba678d86850f3fc97e161f0b13f1596e"}
Jul 22 12:03:32.873617 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:32.873137    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lfqfn" event={"ID":"262cb757-0f60-4d3b-a354-db74abbd3176","Type":"ContainerStarted","Data":"2e540ffdf6bbe973ba56515c2db272093267516a64c824be9abfe0329a16a35f"}
Jul 22 12:03:32.873617 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:32.873292    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:03:32.896492 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:32.896441    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lfqfn" podStartSLOduration=64.731434069 podStartE2EDuration="1m6.896421711s" totalImagesPullingTime="2.164987642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:32.894459561 +0000 UTC m=+101.916481262" watchObservedRunningTime="2026-07-22 12:03:32.896421711 +0000 UTC m=+101.918443416"
Jul 22 12:03:33.792063 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:33.792033    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-48x6v"
Jul 22 12:03:34.336054 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.336019    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:34.338451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.338427    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59d9f112-d3ab-47da-8873-d13561388b67-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cxxpr\" (UID: \"59d9f112-d3ab-47da-8873-d13561388b67\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:34.436864 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.436836    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:34.437047 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.436887    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:34.437513 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.437490    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62903732-d37c-4b43-b094-963c0910ac8b-service-ca-bundle\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:34.439137 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.439119    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62903732-d37c-4b43-b094-963c0910ac8b-metrics-certs\") pod \"router-default-64d58b7fcb-9b8kf\" (UID: \"62903732-d37c-4b43-b094-963c0910ac8b\") " pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:34.595870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.595792    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"
Jul 22 12:03:34.696858 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.696829    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:34.745141 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.745115    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr"]
Jul 22 12:03:34.748367 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:34.748340    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d9f112_d3ab_47da_8873_d13561388b67.slice/crio-d1435def6669fd96d36b621632f72919f7968757bdbc1a0f6840b1218aafaee2 WatchSource:0}: Error finding container d1435def6669fd96d36b621632f72919f7968757bdbc1a0f6840b1218aafaee2: Status 404 returned error can't find the container with id d1435def6669fd96d36b621632f72919f7968757bdbc1a0f6840b1218aafaee2
Jul 22 12:03:34.835195 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.835171    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64d58b7fcb-9b8kf"]
Jul 22 12:03:34.837247 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:34.837219    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62903732_d37c_4b43_b094_963c0910ac8b.slice/crio-ab836855c217135141ecfb242a0043bc89033f1b7ea78410d807d5a5b3392c7e WatchSource:0}: Error finding container ab836855c217135141ecfb242a0043bc89033f1b7ea78410d807d5a5b3392c7e: Status 404 returned error can't find the container with id ab836855c217135141ecfb242a0043bc89033f1b7ea78410d807d5a5b3392c7e
Jul 22 12:03:34.880039 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.880014    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64d58b7fcb-9b8kf" event={"ID":"62903732-d37c-4b43-b094-963c0910ac8b","Type":"ContainerStarted","Data":"ab836855c217135141ecfb242a0043bc89033f1b7ea78410d807d5a5b3392c7e"}
Jul 22 12:03:34.881090 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:34.881066    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr" event={"ID":"59d9f112-d3ab-47da-8873-d13561388b67","Type":"ContainerStarted","Data":"d1435def6669fd96d36b621632f72919f7968757bdbc1a0f6840b1218aafaee2"}
Jul 22 12:03:35.889686 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:35.889651    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64d58b7fcb-9b8kf" event={"ID":"62903732-d37c-4b43-b094-963c0910ac8b","Type":"ContainerStarted","Data":"a5662381e0a6c7799497e812577ae78f342017b8e4e5cd00f03ba49b95085e5c"}
Jul 22 12:03:36.698050 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:36.698019    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:36.700370 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:36.700349    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:36.726333 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:36.726282    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-64d58b7fcb-9b8kf" podStartSLOduration=34.726264892 podStartE2EDuration="34.726264892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:03:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:35.921363222 +0000 UTC m=+104.943384933" watchObservedRunningTime="2026-07-22 12:03:36.726264892 +0000 UTC m=+105.748286595"
Jul 22 12:03:36.896165 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:36.896134    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:36.897460 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:36.897436    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64d58b7fcb-9b8kf"
Jul 22 12:03:37.900300 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:37.900263    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr" event={"ID":"59d9f112-d3ab-47da-8873-d13561388b67","Type":"ContainerStarted","Data":"1241723307a519d30df10c0ef98eed53bec8b5316ed1d0ee6186bd7c588a02f0"}
Jul 22 12:03:37.921396 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:37.921329    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cxxpr" podStartSLOduration=32.889728954 podStartE2EDuration="35.921311587s" totalImagesPullingTime="3.031582633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:37.920588256 +0000 UTC m=+106.942609961" watchObservedRunningTime="2026-07-22 12:03:37.921311587 +0000 UTC m=+106.943333288"
Jul 22 12:03:38.571674 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:38.571643    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:38.573863 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:38.573844    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/01d69365-7cc5-44c0-b707-0f5c2d343d73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hq468\" (UID: \"01d69365-7cc5-44c0-b707-0f5c2d343d73\") " pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:38.585873 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:38.585848    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hq468"
Jul 22 12:03:38.707410 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:38.707373    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hq468"]
Jul 22 12:03:38.709881 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:38.709851    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d69365_7cc5_44c0_b707_0f5c2d343d73.slice/crio-18b2a5002da21be4dd35681eafdff9dc431c385af7a49e600b6b0e96d196f4ec WatchSource:0}: Error finding container 18b2a5002da21be4dd35681eafdff9dc431c385af7a49e600b6b0e96d196f4ec: Status 404 returned error can't find the container with id 18b2a5002da21be4dd35681eafdff9dc431c385af7a49e600b6b0e96d196f4ec
Jul 22 12:03:38.904872 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:38.904840    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hq468" event={"ID":"01d69365-7cc5-44c0-b707-0f5c2d343d73","Type":"ContainerStarted","Data":"18b2a5002da21be4dd35681eafdff9dc431c385af7a49e600b6b0e96d196f4ec"}
Jul 22 12:03:42.878008 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:42.877978    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lfqfn"
Jul 22 12:03:43.924938 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.924892    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hq468" event={"ID":"01d69365-7cc5-44c0-b707-0f5c2d343d73","Type":"ContainerStarted","Data":"4af3287070ce46fdd3fc5884743c418c9bc6b38c41f0fc971792e4cef8d70f47"}
Jul 22 12:03:43.957783 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.957735    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4"]
Jul 22 12:03:43.960816 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.960791    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4"
Jul 22 12:03:43.964564 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.964235    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2kdfq\""
Jul 22 12:03:43.968079 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.965939    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 22 12:03:43.968079 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.966207    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 22 12:03:43.968348 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.968154    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4"]
Jul 22 12:03:43.978410 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.977366    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78446dd59d-8z6fg"]
Jul 22 12:03:43.979766 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.979744    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:43.982875 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.982840    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 22 12:03:43.983039 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.983007    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 22 12:03:43.983988 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.983969    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t9ltr\""
Jul 22 12:03:43.984497 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.984468    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 22 12:03:43.993118 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.993096    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 22 12:03:43.999223 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:43.999166    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78446dd59d-8z6fg"]
Jul 22 12:03:44.114165 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.114117    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ea401827-12e1-4f67-8aa2-8151a4db724c-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-q8cs4\" (UID: \"ea401827-12e1-4f67-8aa2-8151a4db724c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4"
Jul 22 12:03:44.114339 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.114172    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13afccc2-342e-4e10-a82f-883dc273d9d6-installation-pull-secrets\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.114339 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.114200    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13afccc2-342e-4e10-a82f-883dc273d9d6-bound-sa-token\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.114339 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.114299    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13afccc2-342e-4e10-a82f-883dc273d9d6-registry-certificates\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.114799 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.114335    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13afccc2-342e-4e10-a82f-883dc273d9d6-ca-trust-extracted\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.114799 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.114396    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13afccc2-342e-4e10-a82f-883dc273d9d6-registry-tls\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.114799 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.114447    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea401827-12e1-4f67-8aa2-8151a4db724c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-q8cs4\" (UID: \"ea401827-12e1-4f67-8aa2-8151a4db724c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4"
Jul 22 12:03:44.114799 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.114487    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13afccc2-342e-4e10-a82f-883dc273d9d6-trusted-ca\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.114799 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.114525    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13afccc2-342e-4e10-a82f-883dc273d9d6-image-registry-private-configuration\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.114799 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.114552    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngqw5\" (UniqueName: \"kubernetes.io/projected/13afccc2-342e-4e10-a82f-883dc273d9d6-kube-api-access-ngqw5\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.215967 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.215878    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ea401827-12e1-4f67-8aa2-8151a4db724c-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-q8cs4\" (UID: \"ea401827-12e1-4f67-8aa2-8151a4db724c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4"
Jul 22 12:03:44.215967 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.215941    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13afccc2-342e-4e10-a82f-883dc273d9d6-installation-pull-secrets\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.216180 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.215971    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13afccc2-342e-4e10-a82f-883dc273d9d6-bound-sa-token\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.217066 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.217035    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ea401827-12e1-4f67-8aa2-8151a4db724c-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-q8cs4\" (UID: \"ea401827-12e1-4f67-8aa2-8151a4db724c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4"
Jul 22 12:03:44.217293 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.217274    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13afccc2-342e-4e10-a82f-883dc273d9d6-registry-certificates\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.217359 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.217320    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13afccc2-342e-4e10-a82f-883dc273d9d6-ca-trust-extracted\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.217428 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.217364    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13afccc2-342e-4e10-a82f-883dc273d9d6-registry-tls\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.217428 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.217413    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea401827-12e1-4f67-8aa2-8151a4db724c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-q8cs4\" (UID: \"ea401827-12e1-4f67-8aa2-8151a4db724c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4"
Jul 22 12:03:44.217511 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.217444    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13afccc2-342e-4e10-a82f-883dc273d9d6-trusted-ca\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.217511 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.217475    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13afccc2-342e-4e10-a82f-883dc273d9d6-image-registry-private-configuration\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.217511 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.217505    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngqw5\" (UniqueName: \"kubernetes.io/projected/13afccc2-342e-4e10-a82f-883dc273d9d6-kube-api-access-ngqw5\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.222870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.219243    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13afccc2-342e-4e10-a82f-883dc273d9d6-registry-certificates\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.222870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.220113    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13afccc2-342e-4e10-a82f-883dc273d9d6-trusted-ca\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.222870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.220875    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13afccc2-342e-4e10-a82f-883dc273d9d6-ca-trust-extracted\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.224372 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.224344    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea401827-12e1-4f67-8aa2-8151a4db724c-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-q8cs4\" (UID: \"ea401827-12e1-4f67-8aa2-8151a4db724c\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4"
Jul 22 12:03:44.226592 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.226297    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13afccc2-342e-4e10-a82f-883dc273d9d6-installation-pull-secrets\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.226718 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.226701    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13afccc2-342e-4e10-a82f-883dc273d9d6-image-registry-private-configuration\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.227626 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.227597    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13afccc2-342e-4e10-a82f-883dc273d9d6-registry-tls\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.229113 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.229052    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngqw5\" (UniqueName: \"kubernetes.io/projected/13afccc2-342e-4e10-a82f-883dc273d9d6-kube-api-access-ngqw5\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.229356 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.229329    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13afccc2-342e-4e10-a82f-883dc273d9d6-bound-sa-token\") pod \"image-registry-78446dd59d-8z6fg\" (UID: \"13afccc2-342e-4e10-a82f-883dc273d9d6\") " pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.277942 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.277915    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4"
Jul 22 12:03:44.294666 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.294636    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.454135 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.454104    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4"]
Jul 22 12:03:44.457193 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:44.457163    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea401827_12e1_4f67_8aa2_8151a4db724c.slice/crio-5bb3a4697efbdb3422444f2228992b9800849d009535aead40e09284593c9955 WatchSource:0}: Error finding container 5bb3a4697efbdb3422444f2228992b9800849d009535aead40e09284593c9955: Status 404 returned error can't find the container with id 5bb3a4697efbdb3422444f2228992b9800849d009535aead40e09284593c9955
Jul 22 12:03:44.476583 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.476486    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78446dd59d-8z6fg"]
Jul 22 12:03:44.479222 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:44.479194    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13afccc2_342e_4e10_a82f_883dc273d9d6.slice/crio-3ed66eb29e0dcdfdbd2a6758dd0f03fb1e6d497995a002611d0084e7ceedb5bd WatchSource:0}: Error finding container 3ed66eb29e0dcdfdbd2a6758dd0f03fb1e6d497995a002611d0084e7ceedb5bd: Status 404 returned error can't find the container with id 3ed66eb29e0dcdfdbd2a6758dd0f03fb1e6d497995a002611d0084e7ceedb5bd
Jul 22 12:03:44.930148 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.930110    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78446dd59d-8z6fg" event={"ID":"13afccc2-342e-4e10-a82f-883dc273d9d6","Type":"ContainerStarted","Data":"f4a06f83fb534c101fe5ba820c0f0910d044e489b2523d99ef26c4653234bbd9"}
Jul 22 12:03:44.930577 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.930167    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78446dd59d-8z6fg" event={"ID":"13afccc2-342e-4e10-a82f-883dc273d9d6","Type":"ContainerStarted","Data":"3ed66eb29e0dcdfdbd2a6758dd0f03fb1e6d497995a002611d0084e7ceedb5bd"}
Jul 22 12:03:44.930577 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.930221    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:03:44.931567 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.931539    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4" event={"ID":"ea401827-12e1-4f67-8aa2-8151a4db724c","Type":"ContainerStarted","Data":"5bb3a4697efbdb3422444f2228992b9800849d009535aead40e09284593c9955"}
Jul 22 12:03:44.953602 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:44.953552    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78446dd59d-8z6fg" podStartSLOduration=1.953533819 podStartE2EDuration="1.953533819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:03:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:44.952014644 +0000 UTC m=+113.974036345" watchObservedRunningTime="2026-07-22 12:03:44.953533819 +0000 UTC m=+113.975555514"
Jul 22 12:03:46.412486 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.412449    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"]
Jul 22 12:03:46.415678 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.415653    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:46.418668 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.418648    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 22 12:03:46.419993 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.419968    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 12:03:46.420197 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.420177    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 22 12:03:46.420197 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.420194    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-csvms\""
Jul 22 12:03:46.444169 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.444142    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"]
Jul 22 12:03:46.539123 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.539005    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2451b57-6b8f-4135-a644-24362c1344f0-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:46.539123 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.539079    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:46.539363 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.539182    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd9k4\" (UniqueName: \"kubernetes.io/projected/b2451b57-6b8f-4135-a644-24362c1344f0-kube-api-access-nd9k4\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:46.539363 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.539252    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:46.639709 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.639645    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2451b57-6b8f-4135-a644-24362c1344f0-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:46.639888 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.639728    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:46.639888 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.639781    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd9k4\" (UniqueName: \"kubernetes.io/projected/b2451b57-6b8f-4135-a644-24362c1344f0-kube-api-access-nd9k4\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:46.639888 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.639856    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:46.640056 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:46.639975    2584 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 22 12:03:46.640056 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:46.640049    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-tls podName:b2451b57-6b8f-4135-a644-24362c1344f0 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:47.140029477 +0000 UTC m=+116.162051164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-bwz5m" (UID: "b2451b57-6b8f-4135-a644-24362c1344f0") : secret "prometheus-operator-tls" not found
Jul 22 12:03:46.640488 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.640458    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2451b57-6b8f-4135-a644-24362c1344f0-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:46.642754 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.642732    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:46.650312 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:46.650270    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd9k4\" (UniqueName: \"kubernetes.io/projected/b2451b57-6b8f-4135-a644-24362c1344f0-kube-api-access-nd9k4\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:47.143450 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:47.143413    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:47.143642 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:47.143557    2584 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 22 12:03:47.143642 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:47.143624    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-tls podName:b2451b57-6b8f-4135-a644-24362c1344f0 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:48.143609358 +0000 UTC m=+117.165631040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-bwz5m" (UID: "b2451b57-6b8f-4135-a644-24362c1344f0") : secret "prometheus-operator-tls" not found
Jul 22 12:03:47.943674 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:47.943636    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hq468" event={"ID":"01d69365-7cc5-44c0-b707-0f5c2d343d73","Type":"ContainerStarted","Data":"8cfb9f31662c568e752439112c667afb9f0e82064eae1d866d26588a25eab03b"}
Jul 22 12:03:47.945295 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:47.945264    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4" event={"ID":"ea401827-12e1-4f67-8aa2-8151a4db724c","Type":"ContainerStarted","Data":"871184a9819dd5d9281334572b871d1d1928c7d0f20cea2e011a781808b6d531"}
Jul 22 12:03:48.150828 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:48.150772    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:48.153525 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:48.153499    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2451b57-6b8f-4135-a644-24362c1344f0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bwz5m\" (UID: \"b2451b57-6b8f-4135-a644-24362c1344f0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:48.227045 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:48.226967    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"
Jul 22 12:03:48.369755 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:48.369697    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-q8cs4" podStartSLOduration=2.547514123 podStartE2EDuration="5.36967341s" totalImagesPullingTime="2.822159287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:03:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:47.963895045 +0000 UTC m=+116.985916747" watchObservedRunningTime="2026-07-22 12:03:48.36967341 +0000 UTC m=+117.391695112"
Jul 22 12:03:48.370719 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:48.370700    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m"]
Jul 22 12:03:48.378531 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:48.374742    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2451b57_6b8f_4135_a644_24362c1344f0.slice/crio-3c1bef5836bb6064be50c9e665fd668845bbd7ab0ad71b8597a3f09f98de9b1a WatchSource:0}: Error finding container 3c1bef5836bb6064be50c9e665fd668845bbd7ab0ad71b8597a3f09f98de9b1a: Status 404 returned error can't find the container with id 3c1bef5836bb6064be50c9e665fd668845bbd7ab0ad71b8597a3f09f98de9b1a
Jul 22 12:03:48.950613 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:48.950571    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m" event={"ID":"b2451b57-6b8f-4135-a644-24362c1344f0","Type":"ContainerStarted","Data":"3c1bef5836bb6064be50c9e665fd668845bbd7ab0ad71b8597a3f09f98de9b1a"}
Jul 22 12:03:51.962933 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:51.962895    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hq468" event={"ID":"01d69365-7cc5-44c0-b707-0f5c2d343d73","Type":"ContainerStarted","Data":"917d681910f0abf3cf1c5e809784be28d1f5c79b00e7ceaee8cd203e61810c72"}
Jul 22 12:03:51.964581 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:51.964550    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m" event={"ID":"b2451b57-6b8f-4135-a644-24362c1344f0","Type":"ContainerStarted","Data":"f5db7672a981a840465cfb3083c8ec86eef240a1047dc8734222946f05d1aae8"}
Jul 22 12:03:51.964703 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:51.964592    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m" event={"ID":"b2451b57-6b8f-4135-a644-24362c1344f0","Type":"ContainerStarted","Data":"98306d4fef24b84710699de7e7a888cd3c1f2caf38ca3ef2c2e7dc4629041077"}
Jul 22 12:03:51.986152 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:51.986109    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hq468" podStartSLOduration=33.673027691 podStartE2EDuration="45.986095365s" totalImagesPullingTime="12.313067674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:03:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:51.984958986 +0000 UTC m=+121.006980700" watchObservedRunningTime="2026-07-22 12:03:51.986095365 +0000 UTC m=+121.008117065"
Jul 22 12:03:52.006015 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:52.005971    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bwz5m" podStartSLOduration=3.205041148 podStartE2EDuration="6.00595858s" totalImagesPullingTime="2.800917432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:03:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:52.005606095 +0000 UTC m=+121.027627796" watchObservedRunningTime="2026-07-22 12:03:52.00595858 +0000 UTC m=+121.027980281"
Jul 22 12:03:53.849630 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.849597    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"]
Jul 22 12:03:53.855223 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.855203    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:53.857925 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.857905    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 22 12:03:53.859100 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.859079    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 22 12:03:53.859212 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.859083    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 22 12:03:53.859212 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.859122    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-p67wb\""
Jul 22 12:03:53.867857 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.867836    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"]
Jul 22 12:03:53.869795 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.869774    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4kb42"]
Jul 22 12:03:53.873605 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.873585    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.878198 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.878175    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 12:03:53.878567 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.878546    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 12:03:53.878681 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.878642    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fbpwm\""
Jul 22 12:03:53.879146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.879131    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 12:03:53.897726 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.897701    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-accelerators-collector-config\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.897833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.897737    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:53.897833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.897763    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pklsm\" (UniqueName: \"kubernetes.io/projected/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-api-access-pklsm\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:53.897833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.897783    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40c7a4cf-1267-4e3d-ad93-c0d94078b181-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:53.897833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.897828    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-sys\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.897995 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.897846    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:53.897995 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.897872    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-root\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.897995 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.897903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-textfile\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.897995 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.897927    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-tls\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.897995 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.897947    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/40c7a4cf-1267-4e3d-ad93-c0d94078b181-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:53.897995 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.897967    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:53.898177 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.898020    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.898177 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.898048    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-metrics-client-ca\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.898177 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.898074    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94cf7\" (UniqueName: \"kubernetes.io/projected/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-kube-api-access-94cf7\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.898177 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.898092    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-wtmp\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.999454 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999420    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-metrics-client-ca\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.999454 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999458    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94cf7\" (UniqueName: \"kubernetes.io/projected/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-kube-api-access-94cf7\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.999659 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999480    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-wtmp\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.999659 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999593    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-wtmp\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.999659 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999625    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-accelerators-collector-config\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.999794 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999657    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:53.999794 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999687    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklsm\" (UniqueName: \"kubernetes.io/projected/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-api-access-pklsm\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:53.999794 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999708    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40c7a4cf-1267-4e3d-ad93-c0d94078b181-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:53.999794 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999731    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-sys\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:53.999794 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999750    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:53.999794 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999777    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-root\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.000106 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999847    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-sys\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.000106 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:53.999874    2584 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 22 12:03:54.000106 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999890    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-textfile\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.000106 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999916    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-tls\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.000106 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:53.999934    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-tls podName:40c7a4cf-1267-4e3d-ad93-c0d94078b181 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:54.499914356 +0000 UTC m=+123.521936036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-lk7j8" (UID: "40c7a4cf-1267-4e3d-ad93-c0d94078b181") : secret "kube-state-metrics-tls" not found
Jul 22 12:03:54.000106 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:53.999977    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/40c7a4cf-1267-4e3d-ad93-c0d94078b181-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:54.000106 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:53.999989    2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 22 12:03:54.000106 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.000008    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:54.000106 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:54.000040    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-tls podName:1a08d3e8-2aac-4b4a-9e0f-8a19917d765c nodeName:}" failed. No retries permitted until 2026-07-22 12:03:54.500022885 +0000 UTC m=+123.522044589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-tls") pod "node-exporter-4kb42" (UID: "1a08d3e8-2aac-4b4a-9e0f-8a19917d765c") : secret "node-exporter-tls" not found
Jul 22 12:03:54.000106 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.000066    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-root\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.000106 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.000099    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.000654 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.000137    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-metrics-client-ca\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.000654 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.000476    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/40c7a4cf-1267-4e3d-ad93-c0d94078b181-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:54.000654 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.000519    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-accelerators-collector-config\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.000654 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.000526    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-textfile\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.000654 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.000566    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40c7a4cf-1267-4e3d-ad93-c0d94078b181-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:54.000913 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.000673    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:54.002322 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.002303    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:54.002482 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.002305    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.010486 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.010468    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pklsm\" (UniqueName: \"kubernetes.io/projected/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-api-access-pklsm\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:54.011038 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.011019    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94cf7\" (UniqueName: \"kubernetes.io/projected/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-kube-api-access-94cf7\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.503892 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.503862    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:54.504044 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.503903    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-tls\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.504044 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:54.504018    2584 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 22 12:03:54.504153 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:03:54.504085    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-tls podName:40c7a4cf-1267-4e3d-ad93-c0d94078b181 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:55.504067803 +0000 UTC m=+124.526089481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-lk7j8" (UID: "40c7a4cf-1267-4e3d-ad93-c0d94078b181") : secret "kube-state-metrics-tls" not found
Jul 22 12:03:54.506272 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.506250    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a08d3e8-2aac-4b4a-9e0f-8a19917d765c-node-exporter-tls\") pod \"node-exporter-4kb42\" (UID: \"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c\") " pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.781941 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.781861    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4kb42"
Jul 22 12:03:54.794372 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:54.794336    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a08d3e8_2aac_4b4a_9e0f_8a19917d765c.slice/crio-1224bf8489ecae531a75534eed889deb405aa7ce01877637403fad2009fd5960 WatchSource:0}: Error finding container 1224bf8489ecae531a75534eed889deb405aa7ce01877637403fad2009fd5960: Status 404 returned error can't find the container with id 1224bf8489ecae531a75534eed889deb405aa7ce01877637403fad2009fd5960
Jul 22 12:03:54.922995 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.922959    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c497b584-jlc6l"]
Jul 22 12:03:54.931329 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.931297    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:54.934582 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.934555    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 22 12:03:54.934691 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.934560    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mkjgx\""
Jul 22 12:03:54.935211 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.935196    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 22 12:03:54.935259 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.935217    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 22 12:03:54.935507 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.935491    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 22 12:03:54.935659 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.935637    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 22 12:03:54.935768 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.935675    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 22 12:03:54.935959 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.935946    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 22 12:03:54.946547 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.946524    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 22 12:03:54.951294 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.951273    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c497b584-jlc6l"]
Jul 22 12:03:54.975693 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:54.975654    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4kb42" event={"ID":"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c","Type":"ContainerStarted","Data":"1224bf8489ecae531a75534eed889deb405aa7ce01877637403fad2009fd5960"}
Jul 22 12:03:55.008360 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.008334    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-oauth-config\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.008528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.008372    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-trusted-ca-bundle\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.008528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.008406    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-console-config\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.008528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.008423    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65m5l\" (UniqueName: \"kubernetes.io/projected/59964b79-53be-41f8-99b7-cb2bfa466d26-kube-api-access-65m5l\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.008528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.008476    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-oauth-serving-cert\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.008528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.008526    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-service-ca\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.008715 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.008551    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-serving-cert\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.109195 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.109155    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-console-config\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.109195 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.109197    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65m5l\" (UniqueName: \"kubernetes.io/projected/59964b79-53be-41f8-99b7-cb2bfa466d26-kube-api-access-65m5l\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.109424 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.109222    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-oauth-serving-cert\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.109424 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.109262    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-service-ca\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.109424 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.109406    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-serving-cert\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.109689 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.109506    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-oauth-config\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.109689 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.109568    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-trusted-ca-bundle\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.109985 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.109954    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-console-config\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.110116 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.110000    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-service-ca\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.110116 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.110010    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-oauth-serving-cert\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.110322 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.110302    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-trusted-ca-bundle\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.111900 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.111881    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-oauth-config\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.111900 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.111889    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-serving-cert\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.127837 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.127786    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65m5l\" (UniqueName: \"kubernetes.io/projected/59964b79-53be-41f8-99b7-cb2bfa466d26-kube-api-access-65m5l\") pod \"console-5c497b584-jlc6l\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") " pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.242091 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.242052    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:03:55.389618 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.389538    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c497b584-jlc6l"]
Jul 22 12:03:55.392861 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:55.392833    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59964b79_53be_41f8_99b7_cb2bfa466d26.slice/crio-8b833b129c7459abb6c2c8024a8dade99d814c7546eb23ca6d4f626b6f43f6c3 WatchSource:0}: Error finding container 8b833b129c7459abb6c2c8024a8dade99d814c7546eb23ca6d4f626b6f43f6c3: Status 404 returned error can't find the container with id 8b833b129c7459abb6c2c8024a8dade99d814c7546eb23ca6d4f626b6f43f6c3
Jul 22 12:03:55.513770 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.513738    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:55.516147 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.516124    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40c7a4cf-1267-4e3d-ad93-c0d94078b181-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-lk7j8\" (UID: \"40c7a4cf-1267-4e3d-ad93-c0d94078b181\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:55.664319 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.664241    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"
Jul 22 12:03:55.808909 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.808818    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-lk7j8"]
Jul 22 12:03:55.812908 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:03:55.812881    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c7a4cf_1267_4e3d_ad93_c0d94078b181.slice/crio-a7e9629f316bcdcd73d4e8b6c7ff55f555871f6af6f9a973b566c5cd7b0b5c2e WatchSource:0}: Error finding container a7e9629f316bcdcd73d4e8b6c7ff55f555871f6af6f9a973b566c5cd7b0b5c2e: Status 404 returned error can't find the container with id a7e9629f316bcdcd73d4e8b6c7ff55f555871f6af6f9a973b566c5cd7b0b5c2e
Jul 22 12:03:55.981546 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.981460    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8" event={"ID":"40c7a4cf-1267-4e3d-ad93-c0d94078b181","Type":"ContainerStarted","Data":"a7e9629f316bcdcd73d4e8b6c7ff55f555871f6af6f9a973b566c5cd7b0b5c2e"}
Jul 22 12:03:55.982878 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:55.982845    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c497b584-jlc6l" event={"ID":"59964b79-53be-41f8-99b7-cb2bfa466d26","Type":"ContainerStarted","Data":"8b833b129c7459abb6c2c8024a8dade99d814c7546eb23ca6d4f626b6f43f6c3"}
Jul 22 12:03:58.999293 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:58.999191    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c497b584-jlc6l" event={"ID":"59964b79-53be-41f8-99b7-cb2bfa466d26","Type":"ContainerStarted","Data":"83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a"}
Jul 22 12:03:59.001904 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:59.001769    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8" event={"ID":"40c7a4cf-1267-4e3d-ad93-c0d94078b181","Type":"ContainerStarted","Data":"fa9f9af157076c7177f10ed3519a5038c3b3af40153b9527111dbee041975738"}
Jul 22 12:03:59.001904 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:59.001833    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8" event={"ID":"40c7a4cf-1267-4e3d-ad93-c0d94078b181","Type":"ContainerStarted","Data":"dcc33923b486824b20e3891322ec5efc8b87fd9e075f65f47123a6296ac5bb68"}
Jul 22 12:03:59.001904 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:59.001865    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8" event={"ID":"40c7a4cf-1267-4e3d-ad93-c0d94078b181","Type":"ContainerStarted","Data":"9aae555e164c8fee99ed60ef1fd9bba7003a04af45ab49b677b2af5fbeb909c5"}
Jul 22 12:03:59.030422 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:59.030339    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c497b584-jlc6l" podStartSLOduration=1.932554994 podStartE2EDuration="5.030321331s" totalImagesPullingTime="3.097766337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:59.029608594 +0000 UTC m=+128.051630296" watchObservedRunningTime="2026-07-22 12:03:59.030321331 +0000 UTC m=+128.052343032"
Jul 22 12:03:59.060330 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:03:59.060280    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-lk7j8" podStartSLOduration=3.384111755 podStartE2EDuration="6.060262772s" totalImagesPullingTime="2.676151017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:59.059158272 +0000 UTC m=+128.081179997" watchObservedRunningTime="2026-07-22 12:03:59.060262772 +0000 UTC m=+128.082284462"
Jul 22 12:04:00.009067 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.009030    2584 generic.go:356] "Generic (PLEG): container finished" podID="1a08d3e8-2aac-4b4a-9e0f-8a19917d765c" containerID="4913c00406404b34ae7307706412d4f7ef7f4909bb3010c12a78501d7eaddaae" exitCode=0
Jul 22 12:04:00.009483 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.009118    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4kb42" event={"ID":"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c","Type":"ContainerDied","Data":"4913c00406404b34ae7307706412d4f7ef7f4909bb3010c12a78501d7eaddaae"}
Jul 22 12:04:00.321123 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.321006    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:04:00.325134 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.325113    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.329220 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.329192    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 12:04:00.329807 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.329781    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 12:04:00.329937 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.329820    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 12:04:00.330123 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.330105    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 22 12:04:00.330252 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.330111    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gl8m2\""
Jul 22 12:04:00.330252 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.330160    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 12:04:00.331058 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.331038    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dbndrp8ti0490\""
Jul 22 12:04:00.331116 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.331065    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 12:04:00.331312 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.331292    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 12:04:00.331434 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.331311    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 12:04:00.331434 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.331347    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 12:04:00.331434 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.331361    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 12:04:00.331434 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.331366    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 12:04:00.332025 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.331956    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 12:04:00.332589 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.332571    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 12:04:00.336913 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.336892    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 12:04:00.343328 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.339139    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 12:04:00.353543 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.353517    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:04:00.359900 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.359874    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360019 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.359906    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360019 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.359922    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360019 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.359939    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360019 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.359957    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360010    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360042    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360069    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360091    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360116    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360173    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360190    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360205    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360221    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360694 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360277    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360694 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360312    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360694 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360340    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360694 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360362    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360694 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360425    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.360694 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.360468    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnqwl\" (UniqueName: \"kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-kube-api-access-vnqwl\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.460996 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.460962    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.460996 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461000    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461238 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461021    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461238 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461045    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461238 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461070    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461238 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461197    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461465 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461238    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461465 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461271    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461465 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461297    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461465 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461324    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461465 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461357    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461465 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461398    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461465 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461421    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461465 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461448    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461854 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461497    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461854 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461547    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461854 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461588    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461854 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461624    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461854 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461655    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.461854 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461696    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnqwl\" (UniqueName: \"kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-kube-api-access-vnqwl\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.462146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.461927    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.462682 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.462654    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.463214 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.463190    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.464151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.464121    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.464269 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.464168    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.464592 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.464563    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.464678 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.464649    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.464867 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.464848    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.465304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.465276    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.465608 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.465587    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.465922 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.465637    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.467548 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.465706    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.467838 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.465823    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.467977 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.467947    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.468065 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.468011    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.469318 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.468565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.469742 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.469707    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.469815 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.469783    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.470672 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.470656    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.475662 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.475633    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnqwl\" (UniqueName: \"kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-kube-api-access-vnqwl\") pod \"prometheus-k8s-0\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.636447 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.636414    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:00.799110 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:00.799087    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:04:00.801154 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:04:00.801128    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod940a3aa8_c9c1_4abc_9cc8_35d57b21a7fe.slice/crio-fcc2046a313dcb4654307b26666fd8bdf71ef26b04974af9b640a80450998dce WatchSource:0}: Error finding container fcc2046a313dcb4654307b26666fd8bdf71ef26b04974af9b640a80450998dce: Status 404 returned error can't find the container with id fcc2046a313dcb4654307b26666fd8bdf71ef26b04974af9b640a80450998dce
Jul 22 12:04:01.014704 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:01.014614    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4kb42" event={"ID":"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c","Type":"ContainerStarted","Data":"b8622d31bc701955877a535fdb7f46c0f03fddf1eac4d4f2669c566f560a2cb4"}
Jul 22 12:04:01.014704 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:01.014678    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4kb42" event={"ID":"1a08d3e8-2aac-4b4a-9e0f-8a19917d765c","Type":"ContainerStarted","Data":"23dee7e7a74eee3871e9c3ca37c14306330821c1a1a4064d0df2ff6ccd294b6f"}
Jul 22 12:04:01.015843 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:01.015821    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerStarted","Data":"fcc2046a313dcb4654307b26666fd8bdf71ef26b04974af9b640a80450998dce"}
Jul 22 12:04:01.046583 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:01.046534    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4kb42" podStartSLOduration=3.6620411539999997 podStartE2EDuration="8.046519723s" totalImagesPullingTime="4.321944265s" totalInitContainerRuntime="62.534304ms" isStatefulPod=true podCreationTimestamp="2026-07-22 12:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:04:01.046448654 +0000 UTC m=+130.068470355" watchObservedRunningTime="2026-07-22 12:04:01.046519723 +0000 UTC m=+130.068541434"
Jul 22 12:04:01.167927 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:01.167877    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:04:01.170156 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:01.170128    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b5faa4-567b-421f-a050-87e201154841-metrics-certs\") pod \"network-metrics-daemon-zkjd8\" (UID: \"34b5faa4-567b-421f-a050-87e201154841\") " pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:04:01.446446 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:01.446409    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4n945\""
Jul 22 12:04:01.453119 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:01.453095    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkjd8"
Jul 22 12:04:01.602246 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:01.602217    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zkjd8"]
Jul 22 12:04:01.603952 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:04:01.603924    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b5faa4_567b_421f_a050_87e201154841.slice/crio-06faf6ae7c50db0c1a1875d189ce35410b62bd78d23e2c3ee13b8b586e8c230e WatchSource:0}: Error finding container 06faf6ae7c50db0c1a1875d189ce35410b62bd78d23e2c3ee13b8b586e8c230e: Status 404 returned error can't find the container with id 06faf6ae7c50db0c1a1875d189ce35410b62bd78d23e2c3ee13b8b586e8c230e
Jul 22 12:04:02.023022 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:02.022995    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zkjd8" event={"ID":"34b5faa4-567b-421f-a050-87e201154841","Type":"ContainerStarted","Data":"06faf6ae7c50db0c1a1875d189ce35410b62bd78d23e2c3ee13b8b586e8c230e"}
Jul 22 12:04:03.028551 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:03.028514    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zkjd8" event={"ID":"34b5faa4-567b-421f-a050-87e201154841","Type":"ContainerStarted","Data":"d7e5f214c0c89dfc18ee46f98126d80f994242a0914e77fbee541fdd7a57622c"}
Jul 22 12:04:03.028914 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:03.028605    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zkjd8" event={"ID":"34b5faa4-567b-421f-a050-87e201154841","Type":"ContainerStarted","Data":"cf1494ad9b6e58cf3a93ccc9357f23e74b63149fd8b3446be535165a57cd751d"}
Jul 22 12:04:03.030166 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:03.030145    2584 generic.go:356] "Generic (PLEG): container finished" podID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerID="d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50" exitCode=0
Jul 22 12:04:03.030271 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:03.030194    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerDied","Data":"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"}
Jul 22 12:04:03.068821 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:03.068773    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zkjd8" podStartSLOduration=129.825191899 podStartE2EDuration="2m11.068758164s" totalImagesPullingTime="1.243566265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:04:03.067500581 +0000 UTC m=+132.089522283" watchObservedRunningTime="2026-07-22 12:04:03.068758164 +0000 UTC m=+132.090779865"
Jul 22 12:04:05.243101 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:05.243061    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:04:05.243582 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:05.243143    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:04:05.248704 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:05.248680    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:04:05.940506 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:05.940478    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78446dd59d-8z6fg"
Jul 22 12:04:06.049126 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:06.049095    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:04:07.050207 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:07.050165    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerStarted","Data":"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"}
Jul 22 12:04:07.050704 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:07.050257    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerStarted","Data":"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"}
Jul 22 12:04:09.066636 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:09.066602    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerStarted","Data":"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"}
Jul 22 12:04:09.067033 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:09.066685    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerStarted","Data":"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"}
Jul 22 12:04:09.067033 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:09.066710    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerStarted","Data":"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"}
Jul 22 12:04:09.067033 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:09.066737    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerStarted","Data":"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"}
Jul 22 12:04:09.113320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:09.113265    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.936612325 podStartE2EDuration="9.113246309s" totalImagesPullingTime="6.151771922s" totalInitContainerRuntime="24.862062ms" isStatefulPod=false podCreationTimestamp="2026-07-22 12:04:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:04:09.110066535 +0000 UTC m=+138.132088254" watchObservedRunningTime="2026-07-22 12:04:09.113246309 +0000 UTC m=+138.135268010"
Jul 22 12:04:10.636714 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:10.636677    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:04:21.484147 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:21.484115    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c497b584-jlc6l"]
Jul 22 12:04:22.114056 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:22.114020    2584 generic.go:356] "Generic (PLEG): container finished" podID="000c11b2-8abd-49ff-a384-860d4239e130" containerID="51fb9e5ad1f1548046a20a57e60365544136f6857edee293e5ce8bcb534ed506" exitCode=0
Jul 22 12:04:22.114247 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:22.114098    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw" event={"ID":"000c11b2-8abd-49ff-a384-860d4239e130","Type":"ContainerDied","Data":"51fb9e5ad1f1548046a20a57e60365544136f6857edee293e5ce8bcb534ed506"}
Jul 22 12:04:22.114509 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:22.114491    2584 scope.go:117] "RemoveContainer" containerID="51fb9e5ad1f1548046a20a57e60365544136f6857edee293e5ce8bcb534ed506"
Jul 22 12:04:23.119187 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:23.119152    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5k8vw" event={"ID":"000c11b2-8abd-49ff-a384-860d4239e130","Type":"ContainerStarted","Data":"459914b47d999dac96c8f85b34520e118ed5d57d0539e0b0239b45fc2149dc1c"}
Jul 22 12:04:46.509485 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.509394    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c497b584-jlc6l" podUID="59964b79-53be-41f8-99b7-cb2bfa466d26" containerName="console" containerID="cri-o://83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a" gracePeriod=15
Jul 22 12:04:46.755029 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.755006    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:04:46.882818 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.882787    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-console-config\") pod \"59964b79-53be-41f8-99b7-cb2bfa466d26\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") "
Jul 22 12:04:46.882973 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.882857    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-oauth-serving-cert\") pod \"59964b79-53be-41f8-99b7-cb2bfa466d26\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") "
Jul 22 12:04:46.882973 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.882905    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-trusted-ca-bundle\") pod \"59964b79-53be-41f8-99b7-cb2bfa466d26\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") "
Jul 22 12:04:46.882973 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.882930    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-oauth-config\") pod \"59964b79-53be-41f8-99b7-cb2bfa466d26\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") "
Jul 22 12:04:46.882973 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.882960    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65m5l\" (UniqueName: \"kubernetes.io/projected/59964b79-53be-41f8-99b7-cb2bfa466d26-kube-api-access-65m5l\") pod \"59964b79-53be-41f8-99b7-cb2bfa466d26\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") "
Jul 22 12:04:46.883170 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.882990    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-service-ca\") pod \"59964b79-53be-41f8-99b7-cb2bfa466d26\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") "
Jul 22 12:04:46.883170 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.883021    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-serving-cert\") pod \"59964b79-53be-41f8-99b7-cb2bfa466d26\" (UID: \"59964b79-53be-41f8-99b7-cb2bfa466d26\") "
Jul 22 12:04:46.883273 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.883181    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-console-config" (OuterVolumeSpecName: "console-config") pod "59964b79-53be-41f8-99b7-cb2bfa466d26" (UID: "59964b79-53be-41f8-99b7-cb2bfa466d26"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:04:46.883327 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.883312    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "59964b79-53be-41f8-99b7-cb2bfa466d26" (UID: "59964b79-53be-41f8-99b7-cb2bfa466d26"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:04:46.883410 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.883322    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "59964b79-53be-41f8-99b7-cb2bfa466d26" (UID: "59964b79-53be-41f8-99b7-cb2bfa466d26"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:04:46.883484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.883456    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-service-ca" (OuterVolumeSpecName: "service-ca") pod "59964b79-53be-41f8-99b7-cb2bfa466d26" (UID: "59964b79-53be-41f8-99b7-cb2bfa466d26"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:04:46.885350 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.885323    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59964b79-53be-41f8-99b7-cb2bfa466d26-kube-api-access-65m5l" (OuterVolumeSpecName: "kube-api-access-65m5l") pod "59964b79-53be-41f8-99b7-cb2bfa466d26" (UID: "59964b79-53be-41f8-99b7-cb2bfa466d26"). InnerVolumeSpecName "kube-api-access-65m5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:04:46.885452 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.885324    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "59964b79-53be-41f8-99b7-cb2bfa466d26" (UID: "59964b79-53be-41f8-99b7-cb2bfa466d26"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:04:46.885452 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.885428    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "59964b79-53be-41f8-99b7-cb2bfa466d26" (UID: "59964b79-53be-41f8-99b7-cb2bfa466d26"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:04:46.984337 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.984285    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-oauth-serving-cert\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:04:46.984337 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.984328    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-trusted-ca-bundle\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:04:46.984337 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.984338    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-oauth-config\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:04:46.984337 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.984347    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65m5l\" (UniqueName: \"kubernetes.io/projected/59964b79-53be-41f8-99b7-cb2bfa466d26-kube-api-access-65m5l\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:04:46.984337 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.984358    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:04:46.984656 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.984366    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59964b79-53be-41f8-99b7-cb2bfa466d26-console-serving-cert\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:04:46.984656 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:46.984396    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59964b79-53be-41f8-99b7-cb2bfa466d26-console-config\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:04:47.213527 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:47.213430    2584 generic.go:356] "Generic (PLEG): container finished" podID="59964b79-53be-41f8-99b7-cb2bfa466d26" containerID="83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a" exitCode=0
Jul 22 12:04:47.213527 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:47.213477    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c497b584-jlc6l"
Jul 22 12:04:47.213722 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:47.213523    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c497b584-jlc6l" event={"ID":"59964b79-53be-41f8-99b7-cb2bfa466d26","Type":"ContainerDied","Data":"83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a"}
Jul 22 12:04:47.213722 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:47.213578    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c497b584-jlc6l" event={"ID":"59964b79-53be-41f8-99b7-cb2bfa466d26","Type":"ContainerDied","Data":"8b833b129c7459abb6c2c8024a8dade99d814c7546eb23ca6d4f626b6f43f6c3"}
Jul 22 12:04:47.213722 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:47.213621    2584 scope.go:117] "RemoveContainer" containerID="83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a"
Jul 22 12:04:47.224473 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:47.224451    2584 scope.go:117] "RemoveContainer" containerID="83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a"
Jul 22 12:04:47.224773 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:04:47.224753    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a\": container with ID starting with 83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a not found: ID does not exist" containerID="83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a"
Jul 22 12:04:47.224879 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:47.224786    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a"} err="failed to get container status \"83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a\": rpc error: code = NotFound desc = could not find container \"83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a\": container with ID starting with 83c8ed4f23125a02bac3c70355d22d87674775a48c95af1c1f55d461f5db868a not found: ID does not exist"
Jul 22 12:04:47.242233 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:47.242205    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c497b584-jlc6l"]
Jul 22 12:04:47.248084 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:47.248063    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c497b584-jlc6l"]
Jul 22 12:04:47.524735 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:04:47.524639    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59964b79-53be-41f8-99b7-cb2bfa466d26" path="/var/lib/kubelet/pods/59964b79-53be-41f8-99b7-cb2bfa466d26/volumes"
Jul 22 12:05:00.636870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:00.636829    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:00.652432 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:00.652408    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:01.280998 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:01.280969    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:18.700509 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:18.700474    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:05:18.701028 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:18.700840    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy" containerID="cri-o://ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698" gracePeriod=600
Jul 22 12:05:18.701028 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:18.700826    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="prometheus" containerID="cri-o://8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36" gracePeriod=600
Jul 22 12:05:18.701028 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:18.700934    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="config-reloader" containerID="cri-o://0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b" gracePeriod=600
Jul 22 12:05:18.701028 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:18.700863    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="thanos-sidecar" containerID="cri-o://98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249" gracePeriod=600
Jul 22 12:05:18.701028 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:18.700889    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy-web" containerID="cri-o://8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0" gracePeriod=600
Jul 22 12:05:18.701312 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:18.701104    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc" gracePeriod=600
Jul 22 12:05:18.955923 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:18.955855    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.053785 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.053756    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-0\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.053785 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.053784    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-1\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.053998 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.053802    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-kube-rbac-proxy\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.053998 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.053830    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-tls\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.053998 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.053973    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-serving-certs-ca-bundle\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.054146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.054020    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnqwl\" (UniqueName: \"kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-kube-api-access-vnqwl\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.054146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.054059    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.054146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.054096    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.054146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.054124    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-trusted-ca-bundle\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.054325 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.054158    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-2\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.054484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.054117    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:05:19.054918 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.054878    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:05:19.056484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.055126    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-db\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.056484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.055368    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-tls-assets\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.056484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.055446    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config-out\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.056484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.055516    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-web-config\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.056484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.055571    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-metrics-client-certs\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.056484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.055605    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-thanos-prometheus-http-client-file\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.056484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.055753    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-kubelet-serving-ca-bundle\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.056484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.055792    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.056484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.055922    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-metrics-client-ca\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.056484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.055936    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:05:19.057032 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.056593    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:05:19.057032 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.056643    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-grpc-tls\") pod \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\" (UID: \"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe\") "
Jul 22 12:05:19.057236 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.057207    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:05:19.057394 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.057313    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:05:19.058555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.058244    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.058555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.058271    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.058555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.058346    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.058555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.058362    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.058555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.058447    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.058555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.058464    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-metrics-client-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.058555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.058317    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:05:19.058969 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.058644    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:05:19.058969 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.058928    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:05:19.059491 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.059321    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:05:19.064528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.062790    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config" (OuterVolumeSpecName: "config") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:05:19.064528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.062867    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:05:19.064528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.063152    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:05:19.064528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.063186    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:05:19.064528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.063227    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:05:19.064528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.063730    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:05:19.064528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.064488    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-kube-api-access-vnqwl" (OuterVolumeSpecName: "kube-api-access-vnqwl") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "kube-api-access-vnqwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:05:19.064871 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.064799    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config-out" (OuterVolumeSpecName: "config-out") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:05:19.065743 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.065716    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:05:19.072586 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.072559    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-web-config" (OuterVolumeSpecName: "web-config") pod "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" (UID: "940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:05:19.159211 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159175    2584 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-kube-rbac-proxy\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159211 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159202    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnqwl\" (UniqueName: \"kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-kube-api-access-vnqwl\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159211 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159214    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159224    2584 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159233    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159242    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-prometheus-k8s-db\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159251    2584 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-tls-assets\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159259    2584 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-config-out\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159266    2584 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-web-config\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159273    2584 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-metrics-client-certs\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159282    2584 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159291    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159299    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.159451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.159307    2584 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe-secret-grpc-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:05:19.332847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.332759    2584 generic.go:356] "Generic (PLEG): container finished" podID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerID="5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc" exitCode=0
Jul 22 12:05:19.332847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.332785    2584 generic.go:356] "Generic (PLEG): container finished" podID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerID="ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698" exitCode=0
Jul 22 12:05:19.332847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.332795    2584 generic.go:356] "Generic (PLEG): container finished" podID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerID="8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0" exitCode=0
Jul 22 12:05:19.332847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.332802    2584 generic.go:356] "Generic (PLEG): container finished" podID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerID="98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249" exitCode=0
Jul 22 12:05:19.332847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.332810    2584 generic.go:356] "Generic (PLEG): container finished" podID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerID="0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b" exitCode=0
Jul 22 12:05:19.332847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.332816    2584 generic.go:356] "Generic (PLEG): container finished" podID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerID="8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36" exitCode=0
Jul 22 12:05:19.332847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.332846    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.333221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.332841    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerDied","Data":"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"}
Jul 22 12:05:19.333221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.332939    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerDied","Data":"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"}
Jul 22 12:05:19.333221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.332967    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerDied","Data":"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"}
Jul 22 12:05:19.333221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.332991    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerDied","Data":"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"}
Jul 22 12:05:19.333221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.333012    2584 scope.go:117] "RemoveContainer" containerID="5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"
Jul 22 12:05:19.333221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.333024    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerDied","Data":"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"}
Jul 22 12:05:19.333221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.333050    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerDied","Data":"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"}
Jul 22 12:05:19.333221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.333082    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe","Type":"ContainerDied","Data":"fcc2046a313dcb4654307b26666fd8bdf71ef26b04974af9b640a80450998dce"}
Jul 22 12:05:19.341425 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.341410    2584 scope.go:117] "RemoveContainer" containerID="ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"
Jul 22 12:05:19.348672 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.348652    2584 scope.go:117] "RemoveContainer" containerID="8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"
Jul 22 12:05:19.354989 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.354972    2584 scope.go:117] "RemoveContainer" containerID="98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"
Jul 22 12:05:19.360063 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.360040    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:05:19.361891 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.361874    2584 scope.go:117] "RemoveContainer" containerID="0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"
Jul 22 12:05:19.365270 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.365251    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:05:19.368811 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.368791    2584 scope.go:117] "RemoveContainer" containerID="8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"
Jul 22 12:05:19.375767 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.375751    2584 scope.go:117] "RemoveContainer" containerID="d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"
Jul 22 12:05:19.381952 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.381930    2584 scope.go:117] "RemoveContainer" containerID="5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"
Jul 22 12:05:19.382210 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:05:19.382192    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": container with ID starting with 5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc not found: ID does not exist" containerID="5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"
Jul 22 12:05:19.382262 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.382218    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"} err="failed to get container status \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": rpc error: code = NotFound desc = could not find container \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": container with ID starting with 5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc not found: ID does not exist"
Jul 22 12:05:19.382262 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.382238    2584 scope.go:117] "RemoveContainer" containerID="ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"
Jul 22 12:05:19.382503 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:05:19.382486    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": container with ID starting with ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698 not found: ID does not exist" containerID="ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"
Jul 22 12:05:19.382554 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.382508    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"} err="failed to get container status \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": rpc error: code = NotFound desc = could not find container \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": container with ID starting with ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698 not found: ID does not exist"
Jul 22 12:05:19.382554 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.382524    2584 scope.go:117] "RemoveContainer" containerID="8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"
Jul 22 12:05:19.382762 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:05:19.382745    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": container with ID starting with 8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0 not found: ID does not exist" containerID="8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"
Jul 22 12:05:19.382802 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.382768    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"} err="failed to get container status \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": rpc error: code = NotFound desc = could not find container \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": container with ID starting with 8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0 not found: ID does not exist"
Jul 22 12:05:19.382802 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.382783    2584 scope.go:117] "RemoveContainer" containerID="98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"
Jul 22 12:05:19.383019 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:05:19.383003    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": container with ID starting with 98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249 not found: ID does not exist" containerID="98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"
Jul 22 12:05:19.383063 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.383023    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"} err="failed to get container status \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": rpc error: code = NotFound desc = could not find container \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": container with ID starting with 98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249 not found: ID does not exist"
Jul 22 12:05:19.383063 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.383035    2584 scope.go:117] "RemoveContainer" containerID="0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"
Jul 22 12:05:19.383248 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:05:19.383232    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": container with ID starting with 0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b not found: ID does not exist" containerID="0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"
Jul 22 12:05:19.383291 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.383253    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"} err="failed to get container status \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": rpc error: code = NotFound desc = could not find container \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": container with ID starting with 0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b not found: ID does not exist"
Jul 22 12:05:19.383291 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.383267    2584 scope.go:117] "RemoveContainer" containerID="8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"
Jul 22 12:05:19.383516 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:05:19.383500    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": container with ID starting with 8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36 not found: ID does not exist" containerID="8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"
Jul 22 12:05:19.383575 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.383521    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"} err="failed to get container status \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": rpc error: code = NotFound desc = could not find container \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": container with ID starting with 8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36 not found: ID does not exist"
Jul 22 12:05:19.383575 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.383535    2584 scope.go:117] "RemoveContainer" containerID="d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"
Jul 22 12:05:19.383744 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:05:19.383728    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": container with ID starting with d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50 not found: ID does not exist" containerID="d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"
Jul 22 12:05:19.383781 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.383746    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"} err="failed to get container status \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": rpc error: code = NotFound desc = could not find container \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": container with ID starting with d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50 not found: ID does not exist"
Jul 22 12:05:19.383781 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.383758    2584 scope.go:117] "RemoveContainer" containerID="5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"
Jul 22 12:05:19.383972 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.383951    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"} err="failed to get container status \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": rpc error: code = NotFound desc = could not find container \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": container with ID starting with 5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc not found: ID does not exist"
Jul 22 12:05:19.384014 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.383974    2584 scope.go:117] "RemoveContainer" containerID="ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"
Jul 22 12:05:19.384221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.384204    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"} err="failed to get container status \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": rpc error: code = NotFound desc = could not find container \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": container with ID starting with ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698 not found: ID does not exist"
Jul 22 12:05:19.384221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.384220    2584 scope.go:117] "RemoveContainer" containerID="8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"
Jul 22 12:05:19.384510 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.384493    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"} err="failed to get container status \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": rpc error: code = NotFound desc = could not find container \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": container with ID starting with 8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0 not found: ID does not exist"
Jul 22 12:05:19.384564 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.384510    2584 scope.go:117] "RemoveContainer" containerID="98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"
Jul 22 12:05:19.384704 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.384689    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"} err="failed to get container status \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": rpc error: code = NotFound desc = could not find container \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": container with ID starting with 98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249 not found: ID does not exist"
Jul 22 12:05:19.384745 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.384704    2584 scope.go:117] "RemoveContainer" containerID="0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"
Jul 22 12:05:19.384900 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.384884    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"} err="failed to get container status \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": rpc error: code = NotFound desc = could not find container \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": container with ID starting with 0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b not found: ID does not exist"
Jul 22 12:05:19.384952 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.384902    2584 scope.go:117] "RemoveContainer" containerID="8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"
Jul 22 12:05:19.385104 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.385082    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"} err="failed to get container status \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": rpc error: code = NotFound desc = could not find container \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": container with ID starting with 8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36 not found: ID does not exist"
Jul 22 12:05:19.385104 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.385103    2584 scope.go:117] "RemoveContainer" containerID="d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"
Jul 22 12:05:19.385296 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.385275    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"} err="failed to get container status \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": rpc error: code = NotFound desc = could not find container \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": container with ID starting with d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50 not found: ID does not exist"
Jul 22 12:05:19.385296 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.385295    2584 scope.go:117] "RemoveContainer" containerID="5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"
Jul 22 12:05:19.385510 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.385494    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"} err="failed to get container status \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": rpc error: code = NotFound desc = could not find container \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": container with ID starting with 5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc not found: ID does not exist"
Jul 22 12:05:19.385556 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.385510    2584 scope.go:117] "RemoveContainer" containerID="ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"
Jul 22 12:05:19.385703 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.385685    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"} err="failed to get container status \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": rpc error: code = NotFound desc = could not find container \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": container with ID starting with ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698 not found: ID does not exist"
Jul 22 12:05:19.385744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.385704    2584 scope.go:117] "RemoveContainer" containerID="8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"
Jul 22 12:05:19.385907 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.385889    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"} err="failed to get container status \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": rpc error: code = NotFound desc = could not find container \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": container with ID starting with 8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0 not found: ID does not exist"
Jul 22 12:05:19.385983 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.385909    2584 scope.go:117] "RemoveContainer" containerID="98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"
Jul 22 12:05:19.386124 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.386103    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"} err="failed to get container status \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": rpc error: code = NotFound desc = could not find container \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": container with ID starting with 98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249 not found: ID does not exist"
Jul 22 12:05:19.386166 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.386125    2584 scope.go:117] "RemoveContainer" containerID="0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"
Jul 22 12:05:19.386308 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.386291    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"} err="failed to get container status \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": rpc error: code = NotFound desc = could not find container \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": container with ID starting with 0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b not found: ID does not exist"
Jul 22 12:05:19.386395 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.386310    2584 scope.go:117] "RemoveContainer" containerID="8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"
Jul 22 12:05:19.386524 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.386504    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"} err="failed to get container status \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": rpc error: code = NotFound desc = could not find container \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": container with ID starting with 8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36 not found: ID does not exist"
Jul 22 12:05:19.386567 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.386525    2584 scope.go:117] "RemoveContainer" containerID="d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"
Jul 22 12:05:19.386743 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.386725    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"} err="failed to get container status \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": rpc error: code = NotFound desc = could not find container \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": container with ID starting with d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50 not found: ID does not exist"
Jul 22 12:05:19.386787 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.386744    2584 scope.go:117] "RemoveContainer" containerID="5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"
Jul 22 12:05:19.386958 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.386939    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"} err="failed to get container status \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": rpc error: code = NotFound desc = could not find container \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": container with ID starting with 5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc not found: ID does not exist"
Jul 22 12:05:19.386996 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.386959    2584 scope.go:117] "RemoveContainer" containerID="ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"
Jul 22 12:05:19.387137 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.387123    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"} err="failed to get container status \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": rpc error: code = NotFound desc = could not find container \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": container with ID starting with ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698 not found: ID does not exist"
Jul 22 12:05:19.387185 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.387137    2584 scope.go:117] "RemoveContainer" containerID="8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"
Jul 22 12:05:19.387297 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.387284    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"} err="failed to get container status \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": rpc error: code = NotFound desc = could not find container \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": container with ID starting with 8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0 not found: ID does not exist"
Jul 22 12:05:19.387334 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.387298    2584 scope.go:117] "RemoveContainer" containerID="98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"
Jul 22 12:05:19.387506 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.387491    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"} err="failed to get container status \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": rpc error: code = NotFound desc = could not find container \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": container with ID starting with 98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249 not found: ID does not exist"
Jul 22 12:05:19.387566 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.387506    2584 scope.go:117] "RemoveContainer" containerID="0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"
Jul 22 12:05:19.387739 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.387720    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"} err="failed to get container status \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": rpc error: code = NotFound desc = could not find container \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": container with ID starting with 0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b not found: ID does not exist"
Jul 22 12:05:19.387784 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.387740    2584 scope.go:117] "RemoveContainer" containerID="8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"
Jul 22 12:05:19.387943 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.387927    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"} err="failed to get container status \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": rpc error: code = NotFound desc = could not find container \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": container with ID starting with 8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36 not found: ID does not exist"
Jul 22 12:05:19.387989 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.387943    2584 scope.go:117] "RemoveContainer" containerID="d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"
Jul 22 12:05:19.388124 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.388109    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"} err="failed to get container status \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": rpc error: code = NotFound desc = could not find container \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": container with ID starting with d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50 not found: ID does not exist"
Jul 22 12:05:19.388167 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.388124    2584 scope.go:117] "RemoveContainer" containerID="5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"
Jul 22 12:05:19.388262 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.388248    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"} err="failed to get container status \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": rpc error: code = NotFound desc = could not find container \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": container with ID starting with 5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc not found: ID does not exist"
Jul 22 12:05:19.388305 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.388262    2584 scope.go:117] "RemoveContainer" containerID="ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"
Jul 22 12:05:19.388488 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.388472    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"} err="failed to get container status \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": rpc error: code = NotFound desc = could not find container \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": container with ID starting with ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698 not found: ID does not exist"
Jul 22 12:05:19.388541 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.388488    2584 scope.go:117] "RemoveContainer" containerID="8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"
Jul 22 12:05:19.388705 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.388689    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"} err="failed to get container status \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": rpc error: code = NotFound desc = could not find container \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": container with ID starting with 8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0 not found: ID does not exist"
Jul 22 12:05:19.388705 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.388703    2584 scope.go:117] "RemoveContainer" containerID="98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"
Jul 22 12:05:19.388920 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.388903    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"} err="failed to get container status \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": rpc error: code = NotFound desc = could not find container \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": container with ID starting with 98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249 not found: ID does not exist"
Jul 22 12:05:19.388964 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.388921    2584 scope.go:117] "RemoveContainer" containerID="0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"
Jul 22 12:05:19.389125 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.389106    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"} err="failed to get container status \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": rpc error: code = NotFound desc = could not find container \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": container with ID starting with 0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b not found: ID does not exist"
Jul 22 12:05:19.389125 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.389124    2584 scope.go:117] "RemoveContainer" containerID="8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"
Jul 22 12:05:19.389326 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.389299    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"} err="failed to get container status \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": rpc error: code = NotFound desc = could not find container \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": container with ID starting with 8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36 not found: ID does not exist"
Jul 22 12:05:19.389326 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.389323    2584 scope.go:117] "RemoveContainer" containerID="d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"
Jul 22 12:05:19.389576 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.389553    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"} err="failed to get container status \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": rpc error: code = NotFound desc = could not find container \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": container with ID starting with d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50 not found: ID does not exist"
Jul 22 12:05:19.389655 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.389577    2584 scope.go:117] "RemoveContainer" containerID="5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"
Jul 22 12:05:19.389914 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.389890    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc"} err="failed to get container status \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": rpc error: code = NotFound desc = could not find container \"5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc\": container with ID starting with 5f167e7bda0fb61bb4efa66b2d409a43c5ed5fb32a66b2597edcda6984a9bccc not found: ID does not exist"
Jul 22 12:05:19.389914 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.389913    2584 scope.go:117] "RemoveContainer" containerID="ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"
Jul 22 12:05:19.390167 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.390140    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698"} err="failed to get container status \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": rpc error: code = NotFound desc = could not find container \"ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698\": container with ID starting with ebec24e5be4d80f012abf336831cdba596bbaa6fc42725d3f7f09da9f933d698 not found: ID does not exist"
Jul 22 12:05:19.390243 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.390168    2584 scope.go:117] "RemoveContainer" containerID="8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"
Jul 22 12:05:19.390396 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.390362    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0"} err="failed to get container status \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": rpc error: code = NotFound desc = could not find container \"8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0\": container with ID starting with 8d01507f59da879eeeedb53703731ee4c7898787ba1de2140a28a1497a974af0 not found: ID does not exist"
Jul 22 12:05:19.390488 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.390398    2584 scope.go:117] "RemoveContainer" containerID="98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"
Jul 22 12:05:19.390627 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.390609    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249"} err="failed to get container status \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": rpc error: code = NotFound desc = could not find container \"98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249\": container with ID starting with 98d24f8faa261e2d432e175d7788dea5e017fb15e33b58126e1fdfdd0bd1b249 not found: ID does not exist"
Jul 22 12:05:19.390668 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.390629    2584 scope.go:117] "RemoveContainer" containerID="0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"
Jul 22 12:05:19.390851 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.390811    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b"} err="failed to get container status \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": rpc error: code = NotFound desc = could not find container \"0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b\": container with ID starting with 0ca09dd606ae1d8b79306ce04bbf0ef2a85f42e339e95f48053b8ee143ba7e6b not found: ID does not exist"
Jul 22 12:05:19.390895 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.390852    2584 scope.go:117] "RemoveContainer" containerID="8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"
Jul 22 12:05:19.391073 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.391055    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36"} err="failed to get container status \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": rpc error: code = NotFound desc = could not find container \"8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36\": container with ID starting with 8c23aa8bcfd239ecae3fe48dfac9b0d79adaa46157b822b97f8da197acf60d36 not found: ID does not exist"
Jul 22 12:05:19.391130 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.391074    2584 scope.go:117] "RemoveContainer" containerID="d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"
Jul 22 12:05:19.391319 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.391299    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50"} err="failed to get container status \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": rpc error: code = NotFound desc = could not find container \"d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50\": container with ID starting with d9c61ed296033641d49be9a9ed0ab4da38f796342065ab1d9980bf52fa279f50 not found: ID does not exist"
Jul 22 12:05:19.400571 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400548    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:05:19.400841 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400825    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy"
Jul 22 12:05:19.400906 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400843    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy"
Jul 22 12:05:19.400906 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400861    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="prometheus"
Jul 22 12:05:19.400906 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400870    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="prometheus"
Jul 22 12:05:19.400906 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400881    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="thanos-sidecar"
Jul 22 12:05:19.400906 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400890    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="thanos-sidecar"
Jul 22 12:05:19.400906 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400903    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy-thanos"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400911    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy-thanos"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400924    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="init-config-reloader"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400932    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="init-config-reloader"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400942    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59964b79-53be-41f8-99b7-cb2bfa466d26" containerName="console"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400950    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="59964b79-53be-41f8-99b7-cb2bfa466d26" containerName="console"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400968    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy-web"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400976    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy-web"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400985    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="config-reloader"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.400994    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="config-reloader"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.401062    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy-thanos"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.401075    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.401083    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="thanos-sidecar"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.401092    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="59964b79-53be-41f8-99b7-cb2bfa466d26" containerName="console"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.401101    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="kube-rbac-proxy-web"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.401110    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="config-reloader"
Jul 22 12:05:19.401193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.401120    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" containerName="prometheus"
Jul 22 12:05:19.405176 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.405157    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.408372 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408354    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 12:05:19.408833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408569    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 12:05:19.408833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408581    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 12:05:19.408833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408596    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 22 12:05:19.408833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408623    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 12:05:19.408833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408627    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 12:05:19.408833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408650    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dbndrp8ti0490\""
Jul 22 12:05:19.408833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408575    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 12:05:19.408833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408728    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 12:05:19.408833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408794    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gl8m2\""
Jul 22 12:05:19.409290 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408910    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 12:05:19.409290 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408917    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 12:05:19.409290 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.408936    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 12:05:19.409290 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.409236    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 12:05:19.409290 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.409280    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 12:05:19.414568 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.414550    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 12:05:19.414735 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.414720    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 12:05:19.427263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.427240    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:05:19.460614 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.460573    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.460614 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.460611    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d6426c8b-4584-417e-9db0-b7aa3d962374-config-out\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.460771 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.460680    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.460771 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.460712    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.460918 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.460820    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-config\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.460918 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.460875    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.460918 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.460911    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461073 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.460942    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461073 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461007    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461073 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461026    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461073 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461060    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461280 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461083    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461280 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461102    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461280 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461133    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461280 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461188    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d6426c8b-4584-417e-9db0-b7aa3d962374-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461280 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461215    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461280 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461241    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461280 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461264    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-web-config\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461642 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461286    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k26dm\" (UniqueName: \"kubernetes.io/projected/d6426c8b-4584-417e-9db0-b7aa3d962374-kube-api-access-k26dm\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.461642 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.461310    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.524080 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.524046    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe" path="/var/lib/kubelet/pods/940a3aa8-c9c1-4abc-9cc8-35d57b21a7fe/volumes"
Jul 22 12:05:19.561836 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.561807    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.561986 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.561840    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562052 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.561979    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-config\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562052 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562011    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562052 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562029    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562202 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562051    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562202 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562082    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562202 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562106    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562202 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562142    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562202 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562202 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562197    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562535 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562231    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562535 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562286    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d6426c8b-4584-417e-9db0-b7aa3d962374-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562535 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562309    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562535 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562339    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562535 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562363    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-web-config\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562535 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562412    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k26dm\" (UniqueName: \"kubernetes.io/projected/d6426c8b-4584-417e-9db0-b7aa3d962374-kube-api-access-k26dm\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562535 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562437    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562535 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562471    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.562535 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562497    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d6426c8b-4584-417e-9db0-b7aa3d962374-config-out\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.563003 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562771    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.563003 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.562893    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.564490 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.563269    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.564490 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.563353    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.565425 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.564953    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.565425 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.564970    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.565425 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.565009    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d6426c8b-4584-417e-9db0-b7aa3d962374-config-out\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.565425 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.565122    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.565425 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.565353    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.565697 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.565653    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-config\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.566159 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.566093    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.566159 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.566144    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.566331 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.566251    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.566331 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.566277    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.566802 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.566783    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d6426c8b-4584-417e-9db0-b7aa3d962374-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.567253 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.567232    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d6426c8b-4584-417e-9db0-b7aa3d962374-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.567796 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.567776    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.567885 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.567870    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-web-config\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.568286 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.568270    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d6426c8b-4584-417e-9db0-b7aa3d962374-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.575393 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.575364    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k26dm\" (UniqueName: \"kubernetes.io/projected/d6426c8b-4584-417e-9db0-b7aa3d962374-kube-api-access-k26dm\") pod \"prometheus-k8s-0\" (UID: \"d6426c8b-4584-417e-9db0-b7aa3d962374\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.715855 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.715819    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:05:19.858237 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:19.858206    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:05:20.338865 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:20.338833    2584 generic.go:356] "Generic (PLEG): container finished" podID="d6426c8b-4584-417e-9db0-b7aa3d962374" containerID="d5b8e0bb19b3e5460487d33e566ed0ae87a9cd83844099fa6fd62e294fc6e5f1" exitCode=0
Jul 22 12:05:20.339049 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:20.338912    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6426c8b-4584-417e-9db0-b7aa3d962374","Type":"ContainerDied","Data":"d5b8e0bb19b3e5460487d33e566ed0ae87a9cd83844099fa6fd62e294fc6e5f1"}
Jul 22 12:05:20.339049 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:20.338980    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6426c8b-4584-417e-9db0-b7aa3d962374","Type":"ContainerStarted","Data":"b3dd38d1f7290a9fbc5939d219e1bdcc81d6710f344e2834697723722513269a"}
Jul 22 12:05:21.346570 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:21.346532    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6426c8b-4584-417e-9db0-b7aa3d962374","Type":"ContainerStarted","Data":"d300efec40a2b832ecbf42bcf4587731afb6f126f0a8d1c7c8105369194d6391"}
Jul 22 12:05:21.347023 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:21.346624    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6426c8b-4584-417e-9db0-b7aa3d962374","Type":"ContainerStarted","Data":"aaa5c5ad9088547935208f966f71fa754420aff18e15bf8790b54fad2dfdff04"}
Jul 22 12:05:21.347023 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:21.346650    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6426c8b-4584-417e-9db0-b7aa3d962374","Type":"ContainerStarted","Data":"d2cc74735f7f8361e3bf363577f3dc2c59710095f0e2b990c0925d1ecd42d67e"}
Jul 22 12:05:21.347023 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:21.346674    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6426c8b-4584-417e-9db0-b7aa3d962374","Type":"ContainerStarted","Data":"b013f0224e50ff1bd7359e77efa916c1e95bd64e4ebad6a0ea188fd989f7af3a"}
Jul 22 12:05:21.347023 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:21.346718    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6426c8b-4584-417e-9db0-b7aa3d962374","Type":"ContainerStarted","Data":"380240e4f7d3608e1d78444a2763131ecfaa025e5dedece220711805c4d0e76e"}
Jul 22 12:05:21.347023 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:21.346742    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d6426c8b-4584-417e-9db0-b7aa3d962374","Type":"ContainerStarted","Data":"5eab67a2807a1329156ef0613a93639d1e7cb5ec7a50eed42ea7230dc2f1c510"}
Jul 22 12:05:21.380681 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:21.380627    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.360335308 podStartE2EDuration="2.380608126s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.272818ms" isStatefulPod=false podCreationTimestamp="2026-07-22 12:05:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:05:21.379483023 +0000 UTC m=+210.401504746" watchObservedRunningTime="2026-07-22 12:05:21.380608126 +0000 UTC m=+210.402629828"
Jul 22 12:05:24.716091 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:05:24.716060    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:06:19.716659 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:06:19.716607    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:06:19.732035 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:06:19.732006    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:06:20.566281 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:06:20.566250    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:06:51.380476 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:06:51.380445    2584 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 12:07:03.342472 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.342442    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"]
Jul 22 12:07:03.345671 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.345656    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.350065 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.350042    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 12:07:03.350203 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.350123    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 12:07:03.350275 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.350236    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 12:07:03.350329 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.350305    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-47874\""
Jul 22 12:07:03.350397 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.350354    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 12:07:03.363603 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.363582    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"]
Jul 22 12:07:03.416071 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.416036    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/96084ba3-4eb5-4380-ba10-6e2bffc98431-manifests\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.416071 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.416073    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96084ba3-4eb5-4380-ba10-6e2bffc98431-webhook-cert\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.416262 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.416099    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96084ba3-4eb5-4380-ba10-6e2bffc98431-tmp\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.416262 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.416114    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hv5p\" (UniqueName: \"kubernetes.io/projected/96084ba3-4eb5-4380-ba10-6e2bffc98431-kube-api-access-8hv5p\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.416262 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.416197    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/96084ba3-4eb5-4380-ba10-6e2bffc98431-apiservice-cert\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.517169 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.517140    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/96084ba3-4eb5-4380-ba10-6e2bffc98431-manifests\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.517169 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.517176    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96084ba3-4eb5-4380-ba10-6e2bffc98431-webhook-cert\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.517456 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.517217    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96084ba3-4eb5-4380-ba10-6e2bffc98431-tmp\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.517456 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.517239    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hv5p\" (UniqueName: \"kubernetes.io/projected/96084ba3-4eb5-4380-ba10-6e2bffc98431-kube-api-access-8hv5p\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.517456 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.517299    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/96084ba3-4eb5-4380-ba10-6e2bffc98431-apiservice-cert\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.517628 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.517605    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/96084ba3-4eb5-4380-ba10-6e2bffc98431-manifests\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.519495 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.519464    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96084ba3-4eb5-4380-ba10-6e2bffc98431-tmp\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.519715 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.519697    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/96084ba3-4eb5-4380-ba10-6e2bffc98431-apiservice-cert\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.519935 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.519914    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96084ba3-4eb5-4380-ba10-6e2bffc98431-webhook-cert\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.527813 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.527790    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hv5p\" (UniqueName: \"kubernetes.io/projected/96084ba3-4eb5-4380-ba10-6e2bffc98431-kube-api-access-8hv5p\") pod \"opendatahub-operator-controller-manager-979495d87-d4lcn\" (UID: \"96084ba3-4eb5-4380-ba10-6e2bffc98431\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.656471 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.656442    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:03.796450 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.796427    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"]
Jul 22 12:07:03.798983 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:07:03.798956    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96084ba3_4eb5_4380_ba10_6e2bffc98431.slice/crio-fef7ac3dcda80c515fc193235e3fd39ecb3fe186e3976d0de5d37293f0537ca3 WatchSource:0}: Error finding container fef7ac3dcda80c515fc193235e3fd39ecb3fe186e3976d0de5d37293f0537ca3: Status 404 returned error can't find the container with id fef7ac3dcda80c515fc193235e3fd39ecb3fe186e3976d0de5d37293f0537ca3
Jul 22 12:07:03.800843 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:03.800826    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:07:04.712071 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:04.712037    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn" event={"ID":"96084ba3-4eb5-4380-ba10-6e2bffc98431","Type":"ContainerStarted","Data":"fef7ac3dcda80c515fc193235e3fd39ecb3fe186e3976d0de5d37293f0537ca3"}
Jul 22 12:07:09.735553 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:09.735518    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn" event={"ID":"96084ba3-4eb5-4380-ba10-6e2bffc98431","Type":"ContainerStarted","Data":"e2e7ed8c024e1a1beda45639cb07a0469f9c443f5a4c5de2b4273fba50b71dd6"}
Jul 22 12:07:10.740819 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:10.740785    2584 generic.go:356] "Generic (PLEG): container finished" podID="96084ba3-4eb5-4380-ba10-6e2bffc98431" containerID="e2e7ed8c024e1a1beda45639cb07a0469f9c443f5a4c5de2b4273fba50b71dd6" exitCode=0
Jul 22 12:07:10.741169 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:10.740875    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn" event={"ID":"96084ba3-4eb5-4380-ba10-6e2bffc98431","Type":"ContainerDied","Data":"e2e7ed8c024e1a1beda45639cb07a0469f9c443f5a4c5de2b4273fba50b71dd6"}
Jul 22 12:07:11.745973 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:11.745941    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn" event={"ID":"96084ba3-4eb5-4380-ba10-6e2bffc98431","Type":"ContainerStarted","Data":"34d18f91b985f436dadac1cd60163475c10c3071754b55db894a3038604c8f67"}
Jul 22 12:07:11.746332 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:11.746025    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:11.793175 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:11.793129    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn" podStartSLOduration=2.062748936 podStartE2EDuration="8.793114441s" totalImagesPullingTime="5.669377083s" totalInitContainerRuntime="1.060988422s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:07:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:07:11.792036212 +0000 UTC m=+320.814057913" watchObservedRunningTime="2026-07-22 12:07:11.793114441 +0000 UTC m=+320.815136143"
Jul 22 12:07:22.751791 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:22.751759    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-d4lcn"
Jul 22 12:07:31.983550 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:31.983514    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"]
Jul 22 12:07:31.993182 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:31.993159    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:31.998192 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:31.998159    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 22 12:07:31.998813 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:31.998796    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 22 12:07:31.999912 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:31.999897    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zmczj\""
Jul 22 12:07:32.000245 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.000232    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 12:07:32.000589 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.000574    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 22 12:07:32.007956 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.007928    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"]
Jul 22 12:07:32.157691 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.157656    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-istio-token\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.157870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.157699    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.157870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.157726    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.157870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.157783    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-cacerts\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.157870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.157807    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.157870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.157857    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tklds\" (UniqueName: \"kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-kube-api-access-tklds\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.158075 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.157903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9424cd7-eabe-475a-8b4f-075b734be810-local-certs\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.259065 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.258981    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-cacerts\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.259065 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.259018    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.259065 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.259046    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tklds\" (UniqueName: \"kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-kube-api-access-tklds\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.259324 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.259091    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9424cd7-eabe-475a-8b4f-075b734be810-local-certs\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.259324 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.259125    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-istio-token\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.259324 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.259158    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.259324 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.259281    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.259848 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.259823    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.261519 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.261494    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-cacerts\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.261680 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.261662    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.261783 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.261763    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9424cd7-eabe-475a-8b4f-075b734be810-local-certs\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.261847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.261832    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.278370 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.278345    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-istio-token\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.278684 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.278668    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tklds\" (UniqueName: \"kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-kube-api-access-tklds\") pod \"istiod-openshift-gateway-7574459656-m7kcs\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.306646 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.306623    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:32.442184 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.442161    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"]
Jul 22 12:07:32.443852 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:07:32.443824    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9424cd7_eabe_475a_8b4f_075b734be810.slice/crio-6c2b148487b362ed844b80c4c26add02b025774bfe9ca05373826ec8e1e12771 WatchSource:0}: Error finding container 6c2b148487b362ed844b80c4c26add02b025774bfe9ca05373826ec8e1e12771: Status 404 returned error can't find the container with id 6c2b148487b362ed844b80c4c26add02b025774bfe9ca05373826ec8e1e12771
Jul 22 12:07:32.829117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:32.829085    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs" event={"ID":"a9424cd7-eabe-475a-8b4f-075b734be810","Type":"ContainerStarted","Data":"6c2b148487b362ed844b80c4c26add02b025774bfe9ca05373826ec8e1e12771"}
Jul 22 12:07:35.524148 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.524112    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"]
Jul 22 12:07:35.526295 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.526276    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.535264 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.535246    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 22 12:07:35.538258 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.538239    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 22 12:07:35.563611 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.563589    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 22 12:07:35.564748 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.564723    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-svdmj\""
Jul 22 12:07:35.572452 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.572436    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 22 12:07:35.574013 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.573998    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 22 12:07:35.595323 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.595295    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"]
Jul 22 12:07:35.688548 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.688517    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f631573-3147-442a-b1f9-c143db8db5a5-manager-config\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.688548 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.688551    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f631573-3147-442a-b1f9-c143db8db5a5-cert\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.688748 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.688576    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f631573-3147-442a-b1f9-c143db8db5a5-metrics-cert\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.688748 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.688625    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwtfq\" (UniqueName: \"kubernetes.io/projected/9f631573-3147-442a-b1f9-c143db8db5a5-kube-api-access-fwtfq\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.789633 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.789474    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f631573-3147-442a-b1f9-c143db8db5a5-manager-config\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.789633 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.789605    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f631573-3147-442a-b1f9-c143db8db5a5-cert\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.789633 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.789627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f631573-3147-442a-b1f9-c143db8db5a5-metrics-cert\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.789930 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.789648    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwtfq\" (UniqueName: \"kubernetes.io/projected/9f631573-3147-442a-b1f9-c143db8db5a5-kube-api-access-fwtfq\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.790117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.790095    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f631573-3147-442a-b1f9-c143db8db5a5-manager-config\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.792114 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.792093    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f631573-3147-442a-b1f9-c143db8db5a5-metrics-cert\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.792224 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.792173    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f631573-3147-442a-b1f9-c143db8db5a5-cert\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:35.836607 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:35.836578    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwtfq\" (UniqueName: \"kubernetes.io/projected/9f631573-3147-442a-b1f9-c143db8db5a5-kube-api-access-fwtfq\") pod \"lws-controller-manager-5fc976ff6d-q4gfr\" (UID: \"9f631573-3147-442a-b1f9-c143db8db5a5\") " pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:36.135291 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:36.135253    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:36.303670 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:07:36.303637    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f631573_3147_442a_b1f9_c143db8db5a5.slice/crio-e90292c5427e44f20fae6c9c80179c8c0f9a16b8af2f64f0e4542c213b42a992 WatchSource:0}: Error finding container e90292c5427e44f20fae6c9c80179c8c0f9a16b8af2f64f0e4542c213b42a992: Status 404 returned error can't find the container with id e90292c5427e44f20fae6c9c80179c8c0f9a16b8af2f64f0e4542c213b42a992
Jul 22 12:07:36.315355 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:36.315322    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"]
Jul 22 12:07:36.849990 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:36.849932    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr" event={"ID":"9f631573-3147-442a-b1f9-c143db8db5a5","Type":"ContainerStarted","Data":"e90292c5427e44f20fae6c9c80179c8c0f9a16b8af2f64f0e4542c213b42a992"}
Jul 22 12:07:37.943416 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:37.943356    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:07:37.943674 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:37.943448    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:07:38.861432 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:38.861404    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs" event={"ID":"a9424cd7-eabe-475a-8b4f-075b734be810","Type":"ContainerStarted","Data":"c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf"}
Jul 22 12:07:38.861651 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:38.861633    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:38.863114 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:38.863089    2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-m7kcs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 12:07:38.863184 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:38.863141    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs" podUID="a9424cd7-eabe-475a-8b4f-075b734be810" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 12:07:39.867982 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:39.867874    2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-m7kcs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 12:07:39.867982 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:39.867932    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs" podUID="a9424cd7-eabe-475a-8b4f-075b734be810" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 12:07:41.536601 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.536553    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs" podStartSLOduration=5.039314003 podStartE2EDuration="10.536539224s" totalImagesPullingTime="5.497225221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:07:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:07:38.9693397 +0000 UTC m=+347.991361402" watchObservedRunningTime="2026-07-22 12:07:41.536539224 +0000 UTC m=+350.558560925"
Jul 22 12:07:41.537429 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.537395    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"]
Jul 22 12:07:41.537537 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.537520    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs" podUID="a9424cd7-eabe-475a-8b4f-075b734be810" containerName="discovery" containerID="cri-o://c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf" gracePeriod=30
Jul 22 12:07:41.538162 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.538143    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:41.771226 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.771204    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:41.876533 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.876503    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr" event={"ID":"9f631573-3147-442a-b1f9-c143db8db5a5","Type":"ContainerStarted","Data":"3908b8728b168766bedadd9ff3fb8c1ddd530e78a36a4805b6578271b91c8b6b"}
Jul 22 12:07:41.876683 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.876621    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:07:41.877853 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.877830    2584 generic.go:356] "Generic (PLEG): container finished" podID="a9424cd7-eabe-475a-8b4f-075b734be810" containerID="c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf" exitCode=0
Jul 22 12:07:41.877941 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.877862    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs" event={"ID":"a9424cd7-eabe-475a-8b4f-075b734be810","Type":"ContainerDied","Data":"c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf"}
Jul 22 12:07:41.877941 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.877874    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"
Jul 22 12:07:41.877941 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.877894    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs" event={"ID":"a9424cd7-eabe-475a-8b4f-075b734be810","Type":"ContainerDied","Data":"6c2b148487b362ed844b80c4c26add02b025774bfe9ca05373826ec8e1e12771"}
Jul 22 12:07:41.878043 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.877929    2584 scope.go:117] "RemoveContainer" containerID="c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf"
Jul 22 12:07:41.885685 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.885666    2584 scope.go:117] "RemoveContainer" containerID="c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf"
Jul 22 12:07:41.885932 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:07:41.885914    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf\": container with ID starting with c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf not found: ID does not exist" containerID="c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf"
Jul 22 12:07:41.885980 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.885941    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf"} err="failed to get container status \"c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf\": rpc error: code = NotFound desc = could not find container \"c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf\": container with ID starting with c10f3bd37924feb1f38c032aadbe05936750635e071aa5545cda11becbfc48bf not found: ID does not exist"
Jul 22 12:07:41.913414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.913356    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr" podStartSLOduration=1.920940302 podStartE2EDuration="6.91334334s" totalImagesPullingTime="4.992403038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:07:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:07:41.911908425 +0000 UTC m=+350.933930138" watchObservedRunningTime="2026-07-22 12:07:41.91334334 +0000 UTC m=+350.935365041"
Jul 22 12:07:41.950889 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.950865    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-cacerts\") pod \"a9424cd7-eabe-475a-8b4f-075b734be810\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") "
Jul 22 12:07:41.950973 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.950932    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-ca-configmap\") pod \"a9424cd7-eabe-475a-8b4f-075b734be810\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") "
Jul 22 12:07:41.950973 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.950969    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9424cd7-eabe-475a-8b4f-075b734be810-local-certs\") pod \"a9424cd7-eabe-475a-8b4f-075b734be810\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") "
Jul 22 12:07:41.951058 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.950988    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-dns-cert\") pod \"a9424cd7-eabe-475a-8b4f-075b734be810\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") "
Jul 22 12:07:41.951058 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.951015    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-kubeconfig\") pod \"a9424cd7-eabe-475a-8b4f-075b734be810\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") "
Jul 22 12:07:41.951147 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.951061    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tklds\" (UniqueName: \"kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-kube-api-access-tklds\") pod \"a9424cd7-eabe-475a-8b4f-075b734be810\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") "
Jul 22 12:07:41.951147 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.951089    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-istio-token\") pod \"a9424cd7-eabe-475a-8b4f-075b734be810\" (UID: \"a9424cd7-eabe-475a-8b4f-075b734be810\") "
Jul 22 12:07:41.951312 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.951283    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "a9424cd7-eabe-475a-8b4f-075b734be810" (UID: "a9424cd7-eabe-475a-8b4f-075b734be810"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:07:41.953307 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.953280    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9424cd7-eabe-475a-8b4f-075b734be810-local-certs" (OuterVolumeSpecName: "local-certs") pod "a9424cd7-eabe-475a-8b4f-075b734be810" (UID: "a9424cd7-eabe-475a-8b4f-075b734be810"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:07:41.953413 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.953285    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-kube-api-access-tklds" (OuterVolumeSpecName: "kube-api-access-tklds") pod "a9424cd7-eabe-475a-8b4f-075b734be810" (UID: "a9424cd7-eabe-475a-8b4f-075b734be810"). InnerVolumeSpecName "kube-api-access-tklds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:07:41.953516 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.953497    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "a9424cd7-eabe-475a-8b4f-075b734be810" (UID: "a9424cd7-eabe-475a-8b4f-075b734be810"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:07:41.953591 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.953510    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-cacerts" (OuterVolumeSpecName: "cacerts") pod "a9424cd7-eabe-475a-8b4f-075b734be810" (UID: "a9424cd7-eabe-475a-8b4f-075b734be810"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:07:41.953591 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.953578    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "a9424cd7-eabe-475a-8b4f-075b734be810" (UID: "a9424cd7-eabe-475a-8b4f-075b734be810"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:07:41.953704 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:41.953683    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-istio-token" (OuterVolumeSpecName: "istio-token") pod "a9424cd7-eabe-475a-8b4f-075b734be810" (UID: "a9424cd7-eabe-475a-8b4f-075b734be810"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:07:42.052313 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:42.052264    2584 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-kubeconfig\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:07:42.052313 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:42.052306    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tklds\" (UniqueName: \"kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-kube-api-access-tklds\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:07:42.052313 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:42.052321    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9424cd7-eabe-475a-8b4f-075b734be810-istio-token\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:07:42.052561 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:42.052333    2584 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-cacerts\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:07:42.052561 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:42.052344    2584 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-ca-configmap\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:07:42.052561 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:42.052357    2584 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9424cd7-eabe-475a-8b4f-075b734be810-local-certs\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:07:42.052561 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:42.052368    2584 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a9424cd7-eabe-475a-8b4f-075b734be810-istio-csr-dns-cert\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:07:42.218445 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:42.218415    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"]
Jul 22 12:07:42.224500 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:42.224478    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7kcs"]
Jul 22 12:07:43.524413 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:43.524332    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9424cd7-eabe-475a-8b4f-075b734be810" path="/var/lib/kubelet/pods/a9424cd7-eabe-475a-8b4f-075b734be810/volumes"
Jul 22 12:07:52.886757 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:07:52.886728    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5fc976ff6d-q4gfr"
Jul 22 12:08:26.444329 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.444290    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9xgdz"]
Jul 22 12:08:26.444868 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.444664    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9424cd7-eabe-475a-8b4f-075b734be810" containerName="discovery"
Jul 22 12:08:26.444868 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.444680    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9424cd7-eabe-475a-8b4f-075b734be810" containerName="discovery"
Jul 22 12:08:26.444868 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.444766    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9424cd7-eabe-475a-8b4f-075b734be810" containerName="discovery"
Jul 22 12:08:26.450023 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.450002    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9xgdz"
Jul 22 12:08:26.453923 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.453896    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 12:08:26.455781 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.455759    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 12:08:26.457435 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.457418    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-mnsc4\""
Jul 22 12:08:26.459271 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.459246    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9xgdz"]
Jul 22 12:08:26.627117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.627086    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht2sl\" (UniqueName: \"kubernetes.io/projected/0a8c5c29-ddb4-411c-9a6f-d48999f34aa3-kube-api-access-ht2sl\") pod \"kuadrant-operator-catalog-9xgdz\" (UID: \"0a8c5c29-ddb4-411c-9a6f-d48999f34aa3\") " pod="kuadrant-system/kuadrant-operator-catalog-9xgdz"
Jul 22 12:08:26.727725 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.727637    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht2sl\" (UniqueName: \"kubernetes.io/projected/0a8c5c29-ddb4-411c-9a6f-d48999f34aa3-kube-api-access-ht2sl\") pod \"kuadrant-operator-catalog-9xgdz\" (UID: \"0a8c5c29-ddb4-411c-9a6f-d48999f34aa3\") " pod="kuadrant-system/kuadrant-operator-catalog-9xgdz"
Jul 22 12:08:26.737832 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.737807    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht2sl\" (UniqueName: \"kubernetes.io/projected/0a8c5c29-ddb4-411c-9a6f-d48999f34aa3-kube-api-access-ht2sl\") pod \"kuadrant-operator-catalog-9xgdz\" (UID: \"0a8c5c29-ddb4-411c-9a6f-d48999f34aa3\") " pod="kuadrant-system/kuadrant-operator-catalog-9xgdz"
Jul 22 12:08:26.761756 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.761731    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9xgdz"
Jul 22 12:08:26.801507 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.801478    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9xgdz"]
Jul 22 12:08:26.915327 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:26.915296    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9xgdz"]
Jul 22 12:08:26.918653 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:08:26.918628    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a8c5c29_ddb4_411c_9a6f_d48999f34aa3.slice/crio-6caa25b9c74bd85d2fe019197410526ae1e7db98c59f01595e3b436f58ce2f2b WatchSource:0}: Error finding container 6caa25b9c74bd85d2fe019197410526ae1e7db98c59f01595e3b436f58ce2f2b: Status 404 returned error can't find the container with id 6caa25b9c74bd85d2fe019197410526ae1e7db98c59f01595e3b436f58ce2f2b
Jul 22 12:08:27.017621 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:27.017545    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmsrn"]
Jul 22 12:08:27.020698 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:27.020682    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmsrn"
Jul 22 12:08:27.035523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:27.035500    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmsrn"]
Jul 22 12:08:27.053848 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:27.053816    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9xgdz" event={"ID":"0a8c5c29-ddb4-411c-9a6f-d48999f34aa3","Type":"ContainerStarted","Data":"6caa25b9c74bd85d2fe019197410526ae1e7db98c59f01595e3b436f58ce2f2b"}
Jul 22 12:08:27.130343 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:27.130312    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq6zz\" (UniqueName: \"kubernetes.io/projected/fccd99f4-0482-4da0-9bdb-5c337818cf78-kube-api-access-rq6zz\") pod \"kuadrant-operator-catalog-mmsrn\" (UID: \"fccd99f4-0482-4da0-9bdb-5c337818cf78\") " pod="kuadrant-system/kuadrant-operator-catalog-mmsrn"
Jul 22 12:08:27.230762 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:27.230731    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq6zz\" (UniqueName: \"kubernetes.io/projected/fccd99f4-0482-4da0-9bdb-5c337818cf78-kube-api-access-rq6zz\") pod \"kuadrant-operator-catalog-mmsrn\" (UID: \"fccd99f4-0482-4da0-9bdb-5c337818cf78\") " pod="kuadrant-system/kuadrant-operator-catalog-mmsrn"
Jul 22 12:08:27.243628 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:27.243597    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq6zz\" (UniqueName: \"kubernetes.io/projected/fccd99f4-0482-4da0-9bdb-5c337818cf78-kube-api-access-rq6zz\") pod \"kuadrant-operator-catalog-mmsrn\" (UID: \"fccd99f4-0482-4da0-9bdb-5c337818cf78\") " pod="kuadrant-system/kuadrant-operator-catalog-mmsrn"
Jul 22 12:08:27.329833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:27.329755    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmsrn"
Jul 22 12:08:27.459220 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:27.459191    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmsrn"]
Jul 22 12:08:27.462569 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:08:27.462544    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfccd99f4_0482_4da0_9bdb_5c337818cf78.slice/crio-2dc1a8d4cdd91c097e69fd31f57d60bb2fe50a1abfca3e30d79f2af1fbba1410 WatchSource:0}: Error finding container 2dc1a8d4cdd91c097e69fd31f57d60bb2fe50a1abfca3e30d79f2af1fbba1410: Status 404 returned error can't find the container with id 2dc1a8d4cdd91c097e69fd31f57d60bb2fe50a1abfca3e30d79f2af1fbba1410
Jul 22 12:08:28.058762 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:28.058732    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmsrn" event={"ID":"fccd99f4-0482-4da0-9bdb-5c337818cf78","Type":"ContainerStarted","Data":"2dc1a8d4cdd91c097e69fd31f57d60bb2fe50a1abfca3e30d79f2af1fbba1410"}
Jul 22 12:08:30.069339 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:30.069307    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmsrn" event={"ID":"fccd99f4-0482-4da0-9bdb-5c337818cf78","Type":"ContainerStarted","Data":"21a0f3928fa88cb1b587ca8609ec7e4ddb78e805397f18839e0cd69830b2e620"}
Jul 22 12:08:30.089722 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:30.089670    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-mmsrn" podStartSLOduration=2.383694701 podStartE2EDuration="4.089655486s" totalImagesPullingTime="1.705960785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:08:30.089132078 +0000 UTC m=+399.111153779" watchObservedRunningTime="2026-07-22 12:08:30.089655486 +0000 UTC m=+399.111677187"
Jul 22 12:08:31.074836 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:31.074793    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9xgdz" event={"ID":"0a8c5c29-ddb4-411c-9a6f-d48999f34aa3","Type":"ContainerStarted","Data":"461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e"}
Jul 22 12:08:31.075292 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:31.074854    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-9xgdz" podUID="0a8c5c29-ddb4-411c-9a6f-d48999f34aa3" containerName="registry-server" containerID="cri-o://461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e" gracePeriod=2
Jul 22 12:08:31.096108 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:31.096061    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-9xgdz" podStartSLOduration=1.761269492 podStartE2EDuration="5.096046001s" totalImagesPullingTime="3.334776509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:08:31.094711782 +0000 UTC m=+400.116733514" watchObservedRunningTime="2026-07-22 12:08:31.096046001 +0000 UTC m=+400.118067701"
Jul 22 12:08:31.310085 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:31.310058    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9xgdz"
Jul 22 12:08:31.469286 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:31.469261    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht2sl\" (UniqueName: \"kubernetes.io/projected/0a8c5c29-ddb4-411c-9a6f-d48999f34aa3-kube-api-access-ht2sl\") pod \"0a8c5c29-ddb4-411c-9a6f-d48999f34aa3\" (UID: \"0a8c5c29-ddb4-411c-9a6f-d48999f34aa3\") "
Jul 22 12:08:31.471412 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:31.471369    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a8c5c29-ddb4-411c-9a6f-d48999f34aa3-kube-api-access-ht2sl" (OuterVolumeSpecName: "kube-api-access-ht2sl") pod "0a8c5c29-ddb4-411c-9a6f-d48999f34aa3" (UID: "0a8c5c29-ddb4-411c-9a6f-d48999f34aa3"). InnerVolumeSpecName "kube-api-access-ht2sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:08:31.570813 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:31.570783    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht2sl\" (UniqueName: \"kubernetes.io/projected/0a8c5c29-ddb4-411c-9a6f-d48999f34aa3-kube-api-access-ht2sl\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:08:32.080229 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:32.080191    2584 generic.go:356] "Generic (PLEG): container finished" podID="0a8c5c29-ddb4-411c-9a6f-d48999f34aa3" containerID="461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e" exitCode=0
Jul 22 12:08:32.080684 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:32.080244    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9xgdz"
Jul 22 12:08:32.080684 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:32.080246    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9xgdz" event={"ID":"0a8c5c29-ddb4-411c-9a6f-d48999f34aa3","Type":"ContainerDied","Data":"461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e"}
Jul 22 12:08:32.080684 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:32.080348    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9xgdz" event={"ID":"0a8c5c29-ddb4-411c-9a6f-d48999f34aa3","Type":"ContainerDied","Data":"6caa25b9c74bd85d2fe019197410526ae1e7db98c59f01595e3b436f58ce2f2b"}
Jul 22 12:08:32.080684 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:32.080367    2584 scope.go:117] "RemoveContainer" containerID="461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e"
Jul 22 12:08:32.088825 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:32.088807    2584 scope.go:117] "RemoveContainer" containerID="461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e"
Jul 22 12:08:32.089042 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:08:32.089026    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e\": container with ID starting with 461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e not found: ID does not exist" containerID="461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e"
Jul 22 12:08:32.089081 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:32.089049    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e"} err="failed to get container status \"461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e\": rpc error: code = NotFound desc = could not find container \"461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e\": container with ID starting with 461eee63081540ed38a255b306d10195067545d28ed7fabb5151bfdcd80ee69e not found: ID does not exist"
Jul 22 12:08:32.106185 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:32.106159    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9xgdz"]
Jul 22 12:08:32.111927 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:32.111907    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9xgdz"]
Jul 22 12:08:33.524508 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:33.524471    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a8c5c29-ddb4-411c-9a6f-d48999f34aa3" path="/var/lib/kubelet/pods/0a8c5c29-ddb4-411c-9a6f-d48999f34aa3/volumes"
Jul 22 12:08:37.330733 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:37.330703    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-mmsrn"
Jul 22 12:08:37.331108 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:37.330756    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-mmsrn"
Jul 22 12:08:37.353147 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:37.353118    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-mmsrn"
Jul 22 12:08:38.126389 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:38.126351    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-mmsrn"
Jul 22 12:08:59.756256 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:59.756226    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq"]
Jul 22 12:08:59.756750 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:59.756535    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a8c5c29-ddb4-411c-9a6f-d48999f34aa3" containerName="registry-server"
Jul 22 12:08:59.756750 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:59.756545    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a8c5c29-ddb4-411c-9a6f-d48999f34aa3" containerName="registry-server"
Jul 22 12:08:59.756750 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:59.756604    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a8c5c29-ddb4-411c-9a6f-d48999f34aa3" containerName="registry-server"
Jul 22 12:08:59.760801 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:59.760786    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq"
Jul 22 12:08:59.768307 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:59.768282    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-9kqm8\""
Jul 22 12:08:59.790155 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:59.790129    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq"]
Jul 22 12:08:59.806277 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:59.806254    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7wbp\" (UniqueName: \"kubernetes.io/projected/b79c82ae-1508-4ae7-b3ab-1bff041d62a5-kube-api-access-n7wbp\") pod \"authorino-operator-75cc8d5cb7-6zwcq\" (UID: \"b79c82ae-1508-4ae7-b3ab-1bff041d62a5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq"
Jul 22 12:08:59.907738 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:59.907700    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7wbp\" (UniqueName: \"kubernetes.io/projected/b79c82ae-1508-4ae7-b3ab-1bff041d62a5-kube-api-access-n7wbp\") pod \"authorino-operator-75cc8d5cb7-6zwcq\" (UID: \"b79c82ae-1508-4ae7-b3ab-1bff041d62a5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq"
Jul 22 12:08:59.937731 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:08:59.937707    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7wbp\" (UniqueName: \"kubernetes.io/projected/b79c82ae-1508-4ae7-b3ab-1bff041d62a5-kube-api-access-n7wbp\") pod \"authorino-operator-75cc8d5cb7-6zwcq\" (UID: \"b79c82ae-1508-4ae7-b3ab-1bff041d62a5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq"
Jul 22 12:09:00.070652 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:00.070569    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq"
Jul 22 12:09:00.206811 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:00.206791    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq"]
Jul 22 12:09:00.208927 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:09:00.208893    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb79c82ae_1508_4ae7_b3ab_1bff041d62a5.slice/crio-01a781ea98ef1199d3fceb258c497440e933e750bbdf110ebf0a41b39c110d63 WatchSource:0}: Error finding container 01a781ea98ef1199d3fceb258c497440e933e750bbdf110ebf0a41b39c110d63: Status 404 returned error can't find the container with id 01a781ea98ef1199d3fceb258c497440e933e750bbdf110ebf0a41b39c110d63
Jul 22 12:09:01.197517 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:01.197483    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq" event={"ID":"b79c82ae-1508-4ae7-b3ab-1bff041d62a5","Type":"ContainerStarted","Data":"01a781ea98ef1199d3fceb258c497440e933e750bbdf110ebf0a41b39c110d63"}
Jul 22 12:09:02.491890 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:02.491850    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"]
Jul 22 12:09:02.496730 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:02.496707    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:02.501871 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:02.501845    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jzwl2\""
Jul 22 12:09:02.506095 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:02.505758    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"]
Jul 22 12:09:02.637790 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:02.637758    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsmkz\" (UniqueName: \"kubernetes.io/projected/e06edab8-848a-48c4-8937-b608082a4c2e-kube-api-access-fsmkz\") pod \"limitador-operator-controller-manager-859d7b78bd-b6q7k\" (UID: \"e06edab8-848a-48c4-8937-b608082a4c2e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:02.738696 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:02.738652    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsmkz\" (UniqueName: \"kubernetes.io/projected/e06edab8-848a-48c4-8937-b608082a4c2e-kube-api-access-fsmkz\") pod \"limitador-operator-controller-manager-859d7b78bd-b6q7k\" (UID: \"e06edab8-848a-48c4-8937-b608082a4c2e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:02.751556 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:02.751481    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsmkz\" (UniqueName: \"kubernetes.io/projected/e06edab8-848a-48c4-8937-b608082a4c2e-kube-api-access-fsmkz\") pod \"limitador-operator-controller-manager-859d7b78bd-b6q7k\" (UID: \"e06edab8-848a-48c4-8937-b608082a4c2e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:02.810341 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:02.810300    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:03.017421 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:03.017396    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"]
Jul 22 12:09:03.019808 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:09:03.019774    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode06edab8_848a_48c4_8937_b608082a4c2e.slice/crio-4780257f8fc6b496dc64bac122fcd995653315003364a0603dcdc280a26f7574 WatchSource:0}: Error finding container 4780257f8fc6b496dc64bac122fcd995653315003364a0603dcdc280a26f7574: Status 404 returned error can't find the container with id 4780257f8fc6b496dc64bac122fcd995653315003364a0603dcdc280a26f7574
Jul 22 12:09:03.208832 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:03.208796    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k" event={"ID":"e06edab8-848a-48c4-8937-b608082a4c2e","Type":"ContainerStarted","Data":"4780257f8fc6b496dc64bac122fcd995653315003364a0603dcdc280a26f7574"}
Jul 22 12:09:03.210332 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:03.210305    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq" event={"ID":"b79c82ae-1508-4ae7-b3ab-1bff041d62a5","Type":"ContainerStarted","Data":"35145f455dba3f1c97fde04ffe76254233af68df49aeb684dcfbfbddf8fe964c"}
Jul 22 12:09:03.210473 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:03.210366    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq"
Jul 22 12:09:03.239344 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:03.239301    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq" podStartSLOduration=1.515470026 podStartE2EDuration="4.239285573s" totalImagesPullingTime="2.723815547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:09:03.238347013 +0000 UTC m=+432.260368734" watchObservedRunningTime="2026-07-22 12:09:03.239285573 +0000 UTC m=+432.261307274"
Jul 22 12:09:05.185213 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.185133    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw"]
Jul 22 12:09:05.188451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.188435    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw"
Jul 22 12:09:05.191847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.191830    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ljggf\""
Jul 22 12:09:05.191945 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.191831    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 22 12:09:05.211890 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.211866    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw"]
Jul 22 12:09:05.221655 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.221624    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k" event={"ID":"e06edab8-848a-48c4-8937-b608082a4c2e","Type":"ContainerStarted","Data":"24b99f5c96cbae716469ffda89d8180f17c9f837d894e9fea3d44f68cbee826a"}
Jul 22 12:09:05.221792 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.221682    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:05.266086 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.266024    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k" podStartSLOduration=1.411667222 podStartE2EDuration="3.266004599s" totalImagesPullingTime="1.854337377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:09:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:09:05.260693003 +0000 UTC m=+434.282714707" watchObservedRunningTime="2026-07-22 12:09:05.266004599 +0000 UTC m=+434.288026303"
Jul 22 12:09:05.271350 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.271318    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz8c5\" (UniqueName: \"kubernetes.io/projected/d3aa1483-f29a-4b34-8803-6937fc93280a-kube-api-access-jz8c5\") pod \"dns-operator-controller-manager-864dd78c4c-74ctw\" (UID: \"d3aa1483-f29a-4b34-8803-6937fc93280a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw"
Jul 22 12:09:05.372762 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.372730    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz8c5\" (UniqueName: \"kubernetes.io/projected/d3aa1483-f29a-4b34-8803-6937fc93280a-kube-api-access-jz8c5\") pod \"dns-operator-controller-manager-864dd78c4c-74ctw\" (UID: \"d3aa1483-f29a-4b34-8803-6937fc93280a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw"
Jul 22 12:09:05.393705 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.393681    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz8c5\" (UniqueName: \"kubernetes.io/projected/d3aa1483-f29a-4b34-8803-6937fc93280a-kube-api-access-jz8c5\") pod \"dns-operator-controller-manager-864dd78c4c-74ctw\" (UID: \"d3aa1483-f29a-4b34-8803-6937fc93280a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw"
Jul 22 12:09:05.499079 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.498993    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw"
Jul 22 12:09:05.654502 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:05.654477    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw"]
Jul 22 12:09:05.656947 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:09:05.656907    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3aa1483_f29a_4b34_8803_6937fc93280a.slice/crio-6d580d8e33c90a8517c18e1b779631402b1cb7fb8771573951ea568711ade2dd WatchSource:0}: Error finding container 6d580d8e33c90a8517c18e1b779631402b1cb7fb8771573951ea568711ade2dd: Status 404 returned error can't find the container with id 6d580d8e33c90a8517c18e1b779631402b1cb7fb8771573951ea568711ade2dd
Jul 22 12:09:06.228178 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:06.228142    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw" event={"ID":"d3aa1483-f29a-4b34-8803-6937fc93280a","Type":"ContainerStarted","Data":"6d580d8e33c90a8517c18e1b779631402b1cb7fb8771573951ea568711ade2dd"}
Jul 22 12:09:09.533432 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.533403    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"]
Jul 22 12:09:09.536772 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.536752    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:09.542109 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.542091    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wtnkv\""
Jul 22 12:09:09.558042 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.558017    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"]
Jul 22 12:09:09.714199 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.714164    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1230216-cc68-48fd-9d25-cc8ac1022704-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xw6sg\" (UID: \"c1230216-cc68-48fd-9d25-cc8ac1022704\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:09.714356 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.714212    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkzqs\" (UniqueName: \"kubernetes.io/projected/c1230216-cc68-48fd-9d25-cc8ac1022704-kube-api-access-vkzqs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xw6sg\" (UID: \"c1230216-cc68-48fd-9d25-cc8ac1022704\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:09.815579 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.815495    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1230216-cc68-48fd-9d25-cc8ac1022704-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xw6sg\" (UID: \"c1230216-cc68-48fd-9d25-cc8ac1022704\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:09.815579 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.815549    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkzqs\" (UniqueName: \"kubernetes.io/projected/c1230216-cc68-48fd-9d25-cc8ac1022704-kube-api-access-vkzqs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xw6sg\" (UID: \"c1230216-cc68-48fd-9d25-cc8ac1022704\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:09.815842 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.815822    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1230216-cc68-48fd-9d25-cc8ac1022704-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xw6sg\" (UID: \"c1230216-cc68-48fd-9d25-cc8ac1022704\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:09.826123 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.826102    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkzqs\" (UniqueName: \"kubernetes.io/projected/c1230216-cc68-48fd-9d25-cc8ac1022704-kube-api-access-vkzqs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xw6sg\" (UID: \"c1230216-cc68-48fd-9d25-cc8ac1022704\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:09.847193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.847168    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:09.981070 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:09.981045    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"]
Jul 22 12:09:09.983562 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:09:09.983537    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1230216_cc68_48fd_9d25_cc8ac1022704.slice/crio-93aa2e94072f8b066fd7542434905cfb424464debd285ffff8ec0a2aa5542688 WatchSource:0}: Error finding container 93aa2e94072f8b066fd7542434905cfb424464debd285ffff8ec0a2aa5542688: Status 404 returned error can't find the container with id 93aa2e94072f8b066fd7542434905cfb424464debd285ffff8ec0a2aa5542688
Jul 22 12:09:10.247216 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:10.247182    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg" event={"ID":"c1230216-cc68-48fd-9d25-cc8ac1022704","Type":"ContainerStarted","Data":"93aa2e94072f8b066fd7542434905cfb424464debd285ffff8ec0a2aa5542688"}
Jul 22 12:09:11.253045 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:11.253004    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw" event={"ID":"d3aa1483-f29a-4b34-8803-6937fc93280a","Type":"ContainerStarted","Data":"0cae6d5472a86e619787add2123db7dce831283cc504fe1f92b3b72c466fc0cf"}
Jul 22 12:09:11.253504 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:11.253065    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw"
Jul 22 12:09:14.218018 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:14.217990    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6zwcq"
Jul 22 12:09:14.250801 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:14.250749    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw" podStartSLOduration=3.8477826090000002 podStartE2EDuration="9.250734347s" totalImagesPullingTime="5.402951738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:09:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:09:11.289991769 +0000 UTC m=+440.312013493" watchObservedRunningTime="2026-07-22 12:09:14.250734347 +0000 UTC m=+443.272756047"
Jul 22 12:09:16.230286 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:16.230261    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:16.282077 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:16.282045    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg" event={"ID":"c1230216-cc68-48fd-9d25-cc8ac1022704","Type":"ContainerStarted","Data":"73c95a8a8818ef8a952e160a656986e2b3d0ef02878fff98bc7edcf530032190"}
Jul 22 12:09:16.282332 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:16.282310    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:16.308116 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:16.308063    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg" podStartSLOduration=1.5899107049999999 podStartE2EDuration="7.308044164s" totalImagesPullingTime="5.718133459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:09:16.306552245 +0000 UTC m=+445.328573947" watchObservedRunningTime="2026-07-22 12:09:16.308044164 +0000 UTC m=+445.330065867"
Jul 22 12:09:22.259942 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:22.259911    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-74ctw"
Jul 22 12:09:27.288139 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:27.288112    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:29.187836 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.187803    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"]
Jul 22 12:09:29.188343 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.187986    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg" podUID="c1230216-cc68-48fd-9d25-cc8ac1022704" containerName="manager" containerID="cri-o://73c95a8a8818ef8a952e160a656986e2b3d0ef02878fff98bc7edcf530032190" gracePeriod=2
Jul 22 12:09:29.189595 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.189570    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"]
Jul 22 12:09:29.192846 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:09:29.192820    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xw6sg\" is forbidden: User \"system:node:ip-10-0-143-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-107.ec2.internal' and this object" podUID="c1230216-cc68-48fd-9d25-cc8ac1022704" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:29.209061 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.209039    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"]
Jul 22 12:09:29.209278 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.209235    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k" podUID="e06edab8-848a-48c4-8937-b608082a4c2e" containerName="manager" containerID="cri-o://24b99f5c96cbae716469ffda89d8180f17c9f837d894e9fea3d44f68cbee826a" gracePeriod=2
Jul 22 12:09:29.212275 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:09:29.212249    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xw6sg\" is forbidden: User \"system:node:ip-10-0-143-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-107.ec2.internal' and this object" podUID="c1230216-cc68-48fd-9d25-cc8ac1022704" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:29.214300 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:09:29.214268    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b6q7k\" is forbidden: User \"system:node:ip-10-0-143-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-107.ec2.internal' and this object" podUID="e06edab8-848a-48c4-8937-b608082a4c2e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:29.215747 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.215726    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"]
Jul 22 12:09:29.234279 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.234258    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"]
Jul 22 12:09:29.234622 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.234607    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1230216-cc68-48fd-9d25-cc8ac1022704" containerName="manager"
Jul 22 12:09:29.234622 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.234622    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1230216-cc68-48fd-9d25-cc8ac1022704" containerName="manager"
Jul 22 12:09:29.234758 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.234636    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e06edab8-848a-48c4-8937-b608082a4c2e" containerName="manager"
Jul 22 12:09:29.234758 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.234641    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e06edab8-848a-48c4-8937-b608082a4c2e" containerName="manager"
Jul 22 12:09:29.234758 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.234692    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1230216-cc68-48fd-9d25-cc8ac1022704" containerName="manager"
Jul 22 12:09:29.234758 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.234700    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e06edab8-848a-48c4-8937-b608082a4c2e" containerName="manager"
Jul 22 12:09:29.237538 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.237521    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:29.240010 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:09:29.239982    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xw6sg\" is forbidden: User \"system:node:ip-10-0-143-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-107.ec2.internal' and this object" podUID="c1230216-cc68-48fd-9d25-cc8ac1022704" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:29.261047 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.261022    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"]
Jul 22 12:09:29.277890 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.277865    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znk6q\" (UID: \"24ad1891-2480-42e5-a1a5-5d2482c7b0a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:29.278057 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.277952    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p52x9\" (UniqueName: \"kubernetes.io/projected/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-kube-api-access-p52x9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znk6q\" (UID: \"24ad1891-2480-42e5-a1a5-5d2482c7b0a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:29.310051 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:09:29.310006    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b6q7k\" is forbidden: User \"system:node:ip-10-0-143-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-107.ec2.internal' and this object" podUID="e06edab8-848a-48c4-8937-b608082a4c2e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:29.349936 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.349898    2584 generic.go:356] "Generic (PLEG): container finished" podID="c1230216-cc68-48fd-9d25-cc8ac1022704" containerID="73c95a8a8818ef8a952e160a656986e2b3d0ef02878fff98bc7edcf530032190" exitCode=0
Jul 22 12:09:29.351581 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.351555    2584 generic.go:356] "Generic (PLEG): container finished" podID="e06edab8-848a-48c4-8937-b608082a4c2e" containerID="24b99f5c96cbae716469ffda89d8180f17c9f837d894e9fea3d44f68cbee826a" exitCode=0
Jul 22 12:09:29.378812 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.378776    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znk6q\" (UID: \"24ad1891-2480-42e5-a1a5-5d2482c7b0a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:29.378963 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.378823    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p52x9\" (UniqueName: \"kubernetes.io/projected/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-kube-api-access-p52x9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znk6q\" (UID: \"24ad1891-2480-42e5-a1a5-5d2482c7b0a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:29.379142 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.379122    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znk6q\" (UID: \"24ad1891-2480-42e5-a1a5-5d2482c7b0a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:29.395924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.395694    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p52x9\" (UniqueName: \"kubernetes.io/projected/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-kube-api-access-p52x9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znk6q\" (UID: \"24ad1891-2480-42e5-a1a5-5d2482c7b0a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:29.456082 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.456062    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:29.458597 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.458575    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:29.461741 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:09:29.461712    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b6q7k\" is forbidden: User \"system:node:ip-10-0-143-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-107.ec2.internal' and this object" podUID="e06edab8-848a-48c4-8937-b608082a4c2e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:29.468498 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:09:29.468478    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xw6sg\" is forbidden: User \"system:node:ip-10-0-143-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-107.ec2.internal' and this object" podUID="c1230216-cc68-48fd-9d25-cc8ac1022704" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:29.470970 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:09:29.470948    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-b6q7k\" is forbidden: User \"system:node:ip-10-0-143-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-107.ec2.internal' and this object" podUID="e06edab8-848a-48c4-8937-b608082a4c2e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:29.473503 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:09:29.473484    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xw6sg\" is forbidden: User \"system:node:ip-10-0-143-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-107.ec2.internal' and this object" podUID="c1230216-cc68-48fd-9d25-cc8ac1022704" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:29.479778 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.479760    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsmkz\" (UniqueName: \"kubernetes.io/projected/e06edab8-848a-48c4-8937-b608082a4c2e-kube-api-access-fsmkz\") pod \"e06edab8-848a-48c4-8937-b608082a4c2e\" (UID: \"e06edab8-848a-48c4-8937-b608082a4c2e\") "
Jul 22 12:09:29.479854 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.479798    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkzqs\" (UniqueName: \"kubernetes.io/projected/c1230216-cc68-48fd-9d25-cc8ac1022704-kube-api-access-vkzqs\") pod \"c1230216-cc68-48fd-9d25-cc8ac1022704\" (UID: \"c1230216-cc68-48fd-9d25-cc8ac1022704\") "
Jul 22 12:09:29.479893 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.479876    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1230216-cc68-48fd-9d25-cc8ac1022704-extensions-socket-volume\") pod \"c1230216-cc68-48fd-9d25-cc8ac1022704\" (UID: \"c1230216-cc68-48fd-9d25-cc8ac1022704\") "
Jul 22 12:09:29.480354 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.480327    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1230216-cc68-48fd-9d25-cc8ac1022704-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c1230216-cc68-48fd-9d25-cc8ac1022704" (UID: "c1230216-cc68-48fd-9d25-cc8ac1022704"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:09:29.481841 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.481820    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1230216-cc68-48fd-9d25-cc8ac1022704-kube-api-access-vkzqs" (OuterVolumeSpecName: "kube-api-access-vkzqs") pod "c1230216-cc68-48fd-9d25-cc8ac1022704" (UID: "c1230216-cc68-48fd-9d25-cc8ac1022704"). InnerVolumeSpecName "kube-api-access-vkzqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:09:29.481934 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.481856    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e06edab8-848a-48c4-8937-b608082a4c2e-kube-api-access-fsmkz" (OuterVolumeSpecName: "kube-api-access-fsmkz") pod "e06edab8-848a-48c4-8937-b608082a4c2e" (UID: "e06edab8-848a-48c4-8937-b608082a4c2e"). InnerVolumeSpecName "kube-api-access-fsmkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:09:29.524827 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.524801    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1230216-cc68-48fd-9d25-cc8ac1022704" path="/var/lib/kubelet/pods/c1230216-cc68-48fd-9d25-cc8ac1022704/volumes"
Jul 22 12:09:29.525196 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.525180    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e06edab8-848a-48c4-8937-b608082a4c2e" path="/var/lib/kubelet/pods/e06edab8-848a-48c4-8937-b608082a4c2e/volumes"
Jul 22 12:09:29.525538 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.525525    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"]
Jul 22 12:09:29.529009 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.528994    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:09:29.550705 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.550680    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"]
Jul 22 12:09:29.580964 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.580935    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t77n\" (UniqueName: \"kubernetes.io/projected/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-kube-api-access-2t77n\") pod \"kuadrant-operator-controller-manager-86878c769f-c9cqb\" (UID: \"eb9a229d-c58f-4b37-9083-aa80f5f4a81e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:09:29.580964 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.580965    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c9cqb\" (UID: \"eb9a229d-c58f-4b37-9083-aa80f5f4a81e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:09:29.581101 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.581061    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1230216-cc68-48fd-9d25-cc8ac1022704-extensions-socket-volume\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:09:29.581101 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.581078    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsmkz\" (UniqueName: \"kubernetes.io/projected/e06edab8-848a-48c4-8937-b608082a4c2e-kube-api-access-fsmkz\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:09:29.581101 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.581088    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkzqs\" (UniqueName: \"kubernetes.io/projected/c1230216-cc68-48fd-9d25-cc8ac1022704-kube-api-access-vkzqs\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:09:29.615609 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.615581    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:29.681621 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.681591    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t77n\" (UniqueName: \"kubernetes.io/projected/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-kube-api-access-2t77n\") pod \"kuadrant-operator-controller-manager-86878c769f-c9cqb\" (UID: \"eb9a229d-c58f-4b37-9083-aa80f5f4a81e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:09:29.681746 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.681632    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c9cqb\" (UID: \"eb9a229d-c58f-4b37-9083-aa80f5f4a81e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:09:29.682014 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.681993    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c9cqb\" (UID: \"eb9a229d-c58f-4b37-9083-aa80f5f4a81e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:09:29.695582 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.695564    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t77n\" (UniqueName: \"kubernetes.io/projected/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-kube-api-access-2t77n\") pod \"kuadrant-operator-controller-manager-86878c769f-c9cqb\" (UID: \"eb9a229d-c58f-4b37-9083-aa80f5f4a81e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:09:29.758047 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.758025    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"]
Jul 22 12:09:29.760028 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:09:29.760003    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ad1891_2480_42e5_a1a5_5d2482c7b0a9.slice/crio-22d0f0f2312df3247b83d4047075db65425721a32a76a2f0c38113bb99a48baa WatchSource:0}: Error finding container 22d0f0f2312df3247b83d4047075db65425721a32a76a2f0c38113bb99a48baa: Status 404 returned error can't find the container with id 22d0f0f2312df3247b83d4047075db65425721a32a76a2f0c38113bb99a48baa
Jul 22 12:09:29.841827 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.841800    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:09:29.982439 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:29.982415    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"]
Jul 22 12:09:29.984441 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:09:29.984415    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb9a229d_c58f_4b37_9083_aa80f5f4a81e.slice/crio-6ba5dd19970444f45811a3a26b42c274efd6f29534331bfa8f98921a0ca15a8d WatchSource:0}: Error finding container 6ba5dd19970444f45811a3a26b42c274efd6f29534331bfa8f98921a0ca15a8d: Status 404 returned error can't find the container with id 6ba5dd19970444f45811a3a26b42c274efd6f29534331bfa8f98921a0ca15a8d
Jul 22 12:09:30.356858 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.356837    2584 scope.go:117] "RemoveContainer" containerID="24b99f5c96cbae716469ffda89d8180f17c9f837d894e9fea3d44f68cbee826a"
Jul 22 12:09:30.357239 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.356838    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b6q7k"
Jul 22 12:09:30.358934 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.358785    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb" event={"ID":"eb9a229d-c58f-4b37-9083-aa80f5f4a81e","Type":"ContainerStarted","Data":"6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9"}
Jul 22 12:09:30.359057 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.358969    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:09:30.359057 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.359003    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb" event={"ID":"eb9a229d-c58f-4b37-9083-aa80f5f4a81e","Type":"ContainerStarted","Data":"6ba5dd19970444f45811a3a26b42c274efd6f29534331bfa8f98921a0ca15a8d"}
Jul 22 12:09:30.360529 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.360508    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q" event={"ID":"24ad1891-2480-42e5-a1a5-5d2482c7b0a9","Type":"ContainerStarted","Data":"a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c"}
Jul 22 12:09:30.360638 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.360551    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:30.360638 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.360576    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q" event={"ID":"24ad1891-2480-42e5-a1a5-5d2482c7b0a9","Type":"ContainerStarted","Data":"22d0f0f2312df3247b83d4047075db65425721a32a76a2f0c38113bb99a48baa"}
Jul 22 12:09:30.361805 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.361789    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xw6sg"
Jul 22 12:09:30.366061 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.366045    2584 scope.go:117] "RemoveContainer" containerID="73c95a8a8818ef8a952e160a656986e2b3d0ef02878fff98bc7edcf530032190"
Jul 22 12:09:30.408102 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.408060    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb" podStartSLOduration=1.40804735 podStartE2EDuration="1.40804735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:09:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:09:30.40704144 +0000 UTC m=+459.429063155" watchObservedRunningTime="2026-07-22 12:09:30.40804735 +0000 UTC m=+459.430069050"
Jul 22 12:09:30.455330 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:30.455280    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q" podStartSLOduration=1.455260779 podStartE2EDuration="1.455260779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:09:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:09:30.454730681 +0000 UTC m=+459.476752381" watchObservedRunningTime="2026-07-22 12:09:30.455260779 +0000 UTC m=+459.477282480"
Jul 22 12:09:41.371315 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:41.371284    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:41.371754 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:41.371354    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:09:41.460589 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:41.460562    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"]
Jul 22 12:09:41.460756 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:41.460737    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q" podUID="24ad1891-2480-42e5-a1a5-5d2482c7b0a9" containerName="manager" containerID="cri-o://a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c" gracePeriod=10
Jul 22 12:09:41.702222 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:41.702200    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:41.784022 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:41.783996    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-extensions-socket-volume\") pod \"24ad1891-2480-42e5-a1a5-5d2482c7b0a9\" (UID: \"24ad1891-2480-42e5-a1a5-5d2482c7b0a9\") "
Jul 22 12:09:41.784159 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:41.784081    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p52x9\" (UniqueName: \"kubernetes.io/projected/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-kube-api-access-p52x9\") pod \"24ad1891-2480-42e5-a1a5-5d2482c7b0a9\" (UID: \"24ad1891-2480-42e5-a1a5-5d2482c7b0a9\") "
Jul 22 12:09:41.784422 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:41.784368    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "24ad1891-2480-42e5-a1a5-5d2482c7b0a9" (UID: "24ad1891-2480-42e5-a1a5-5d2482c7b0a9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:09:41.785994 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:41.785972    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-kube-api-access-p52x9" (OuterVolumeSpecName: "kube-api-access-p52x9") pod "24ad1891-2480-42e5-a1a5-5d2482c7b0a9" (UID: "24ad1891-2480-42e5-a1a5-5d2482c7b0a9"). InnerVolumeSpecName "kube-api-access-p52x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:09:41.885330 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:41.885259    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p52x9\" (UniqueName: \"kubernetes.io/projected/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-kube-api-access-p52x9\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:09:41.885330 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:41.885284    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24ad1891-2480-42e5-a1a5-5d2482c7b0a9-extensions-socket-volume\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:09:42.417676 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:42.417638    2584 generic.go:356] "Generic (PLEG): container finished" podID="24ad1891-2480-42e5-a1a5-5d2482c7b0a9" containerID="a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c" exitCode=0
Jul 22 12:09:42.418141 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:42.417698    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"
Jul 22 12:09:42.418141 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:42.417726    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q" event={"ID":"24ad1891-2480-42e5-a1a5-5d2482c7b0a9","Type":"ContainerDied","Data":"a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c"}
Jul 22 12:09:42.418141 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:42.417830    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q" event={"ID":"24ad1891-2480-42e5-a1a5-5d2482c7b0a9","Type":"ContainerDied","Data":"22d0f0f2312df3247b83d4047075db65425721a32a76a2f0c38113bb99a48baa"}
Jul 22 12:09:42.418141 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:42.417863    2584 scope.go:117] "RemoveContainer" containerID="a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c"
Jul 22 12:09:42.427058 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:42.427041    2584 scope.go:117] "RemoveContainer" containerID="a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c"
Jul 22 12:09:42.427364 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:09:42.427342    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c\": container with ID starting with a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c not found: ID does not exist" containerID="a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c"
Jul 22 12:09:42.427434 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:42.427371    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c"} err="failed to get container status \"a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c\": rpc error: code = NotFound desc = could not find container \"a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c\": container with ID starting with a5143fd8d51a6bc51037d44788bb7f3f5bb0ad1cd702d72df959fe3d2faa950c not found: ID does not exist"
Jul 22 12:09:42.450855 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:42.450832    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"]
Jul 22 12:09:42.454256 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:42.454237    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znk6q"]
Jul 22 12:09:43.524983 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:43.524950    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24ad1891-2480-42e5-a1a5-5d2482c7b0a9" path="/var/lib/kubelet/pods/24ad1891-2480-42e5-a1a5-5d2482c7b0a9/volumes"
Jul 22 12:09:58.034228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.034192    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"]
Jul 22 12:09:58.034618 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.034567    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24ad1891-2480-42e5-a1a5-5d2482c7b0a9" containerName="manager"
Jul 22 12:09:58.034618 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.034580    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="24ad1891-2480-42e5-a1a5-5d2482c7b0a9" containerName="manager"
Jul 22 12:09:58.034691 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.034640    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="24ad1891-2480-42e5-a1a5-5d2482c7b0a9" containerName="manager"
Jul 22 12:09:58.038504 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.038484    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.043787 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.043765    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 12:09:58.043912 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.043765    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-67jqc\""
Jul 22 12:09:58.059542 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.059519    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"]
Jul 22 12:09:58.126061 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.126007    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.126061 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.126054    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.126253 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.126107    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.126253 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.126135    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.126253 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.126190    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.126253 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.126222    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.126253 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.126250    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.126473 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.126272    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.126473 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.126312    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snqpm\" (UniqueName: \"kubernetes.io/projected/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-kube-api-access-snqpm\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.227598 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.227564    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.227598 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.227604    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.227801 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.227620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.227801 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.227643    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.227801 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.227668    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.227801 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.227772    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snqpm\" (UniqueName: \"kubernetes.io/projected/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-kube-api-access-snqpm\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.228003 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.227868    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.228003 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.227897    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.228003 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.227964    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.228003 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.227976    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.228199 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.228011    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.228258 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.228225    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.228318 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.228279    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.228666 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.228645    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.229813 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.229792    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.230333 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.230312    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.249498 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.249473    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.249588 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.249565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snqpm\" (UniqueName: \"kubernetes.io/projected/e5c60d29-9b5e-4b24-a0ac-4e8d118c2270-kube-api-access-snqpm\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fhvks\" (UID: \"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.348091 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.348067    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:09:58.509340 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:58.509315    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"]
Jul 22 12:09:58.511483 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:09:58.511447    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c60d29_9b5e_4b24_a0ac_4e8d118c2270.slice/crio-06a5b9c76856bf9a878a35b0bfa947400955558f7af929c88c514c9c87671ee1 WatchSource:0}: Error finding container 06a5b9c76856bf9a878a35b0bfa947400955558f7af929c88c514c9c87671ee1: Status 404 returned error can't find the container with id 06a5b9c76856bf9a878a35b0bfa947400955558f7af929c88c514c9c87671ee1
Jul 22 12:09:59.495233 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:09:59.495196    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks" event={"ID":"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270","Type":"ContainerStarted","Data":"06a5b9c76856bf9a878a35b0bfa947400955558f7af929c88c514c9c87671ee1"}
Jul 22 12:10:01.163571 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:01.163532    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:10:01.163853 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:01.163605    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:10:01.163853 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:01.163634    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:10:01.505883 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:01.505797    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks" event={"ID":"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270","Type":"ContainerStarted","Data":"6c0962cb00db2429bc57fd7577df1dba1c2b46b5dcdb1b76694970e297e24107"}
Jul 22 12:10:01.534175 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:01.534130    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks" podStartSLOduration=0.884076744 podStartE2EDuration="3.534108375s" totalImagesPullingTime="2.650031631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:10:01.532166177 +0000 UTC m=+490.554187879" watchObservedRunningTime="2026-07-22 12:10:01.534108375 +0000 UTC m=+490.556130080"
Jul 22 12:10:02.348742 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.348710    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:10:02.353623 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.353604    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:10:02.509994 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.509966    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:10:02.511014 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.510992    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:10:02.845124 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.845035    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"]
Jul 22 12:10:02.848037 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.848014    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:02.851085 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.851068    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-l7xw8\""
Jul 22 12:10:02.851341 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.851323    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 22 12:10:02.895726 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.895700    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"]
Jul 22 12:10:02.925070 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.925030    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"]
Jul 22 12:10:02.974204 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.974177    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-config-file\") pod \"limitador-limitador-64c94fd64f-mvbzj\" (UID: \"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:02.974360 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:02.974214    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frsst\" (UniqueName: \"kubernetes.io/projected/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-kube-api-access-frsst\") pod \"limitador-limitador-64c94fd64f-mvbzj\" (UID: \"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:03.074772 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:03.074735    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-config-file\") pod \"limitador-limitador-64c94fd64f-mvbzj\" (UID: \"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:03.074772 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:03.074774    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frsst\" (UniqueName: \"kubernetes.io/projected/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-kube-api-access-frsst\") pod \"limitador-limitador-64c94fd64f-mvbzj\" (UID: \"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:03.075353 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:03.075332    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-config-file\") pod \"limitador-limitador-64c94fd64f-mvbzj\" (UID: \"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:03.085484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:03.085453    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frsst\" (UniqueName: \"kubernetes.io/projected/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-kube-api-access-frsst\") pod \"limitador-limitador-64c94fd64f-mvbzj\" (UID: \"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:03.158874 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:03.158838    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:03.303539 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:03.303515    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"]
Jul 22 12:10:03.305764 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:10:03.305724    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c50876e_87ce_4bfa_a1ab_cdb1fe80f921.slice/crio-09ade8beebc547b926c8ec338699ecc5081bd0a5abf6adb75752aa7513192671 WatchSource:0}: Error finding container 09ade8beebc547b926c8ec338699ecc5081bd0a5abf6adb75752aa7513192671: Status 404 returned error can't find the container with id 09ade8beebc547b926c8ec338699ecc5081bd0a5abf6adb75752aa7513192671
Jul 22 12:10:03.515477 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:03.515397    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj" event={"ID":"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921","Type":"ContainerStarted","Data":"09ade8beebc547b926c8ec338699ecc5081bd0a5abf6adb75752aa7513192671"}
Jul 22 12:10:06.536136 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:06.536045    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj" event={"ID":"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921","Type":"ContainerStarted","Data":"75e332697ac671d0ef8e99d22d4c19c79003fccf597337b601ddcfcc4045134a"}
Jul 22 12:10:06.536136 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:06.536118    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:06.560924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:06.560874    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj" podStartSLOduration=1.683843464 podStartE2EDuration="4.560860803s" totalImagesPullingTime="2.877017339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:10:06.560021703 +0000 UTC m=+495.582043414" watchObservedRunningTime="2026-07-22 12:10:06.560860803 +0000 UTC m=+495.582882504"
Jul 22 12:10:17.540250 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:17.540222    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:18.156278 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:18.156245    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"]
Jul 22 12:10:18.156499 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:18.156456    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj" podUID="9c50876e-87ce-4bfa-a1ab-cdb1fe80f921" containerName="limitador" containerID="cri-o://75e332697ac671d0ef8e99d22d4c19c79003fccf597337b601ddcfcc4045134a" gracePeriod=30
Jul 22 12:10:18.585602 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:18.585572    2584 generic.go:356] "Generic (PLEG): container finished" podID="9c50876e-87ce-4bfa-a1ab-cdb1fe80f921" containerID="75e332697ac671d0ef8e99d22d4c19c79003fccf597337b601ddcfcc4045134a" exitCode=0
Jul 22 12:10:18.585967 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:18.585643    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj" event={"ID":"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921","Type":"ContainerDied","Data":"75e332697ac671d0ef8e99d22d4c19c79003fccf597337b601ddcfcc4045134a"}
Jul 22 12:10:19.096604 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.096581    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:19.212708 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.212676    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frsst\" (UniqueName: \"kubernetes.io/projected/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-kube-api-access-frsst\") pod \"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921\" (UID: \"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921\") "
Jul 22 12:10:19.212884 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.212743    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-config-file\") pod \"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921\" (UID: \"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921\") "
Jul 22 12:10:19.213093 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.213071    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-config-file" (OuterVolumeSpecName: "config-file") pod "9c50876e-87ce-4bfa-a1ab-cdb1fe80f921" (UID: "9c50876e-87ce-4bfa-a1ab-cdb1fe80f921"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:10:19.214792 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.214770    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-kube-api-access-frsst" (OuterVolumeSpecName: "kube-api-access-frsst") pod "9c50876e-87ce-4bfa-a1ab-cdb1fe80f921" (UID: "9c50876e-87ce-4bfa-a1ab-cdb1fe80f921"). InnerVolumeSpecName "kube-api-access-frsst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:10:19.313691 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.313654    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frsst\" (UniqueName: \"kubernetes.io/projected/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-kube-api-access-frsst\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:10:19.313691 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.313685    2584 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921-config-file\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:10:19.591173 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.591149    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"
Jul 22 12:10:19.591173 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.591169    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mvbzj" event={"ID":"9c50876e-87ce-4bfa-a1ab-cdb1fe80f921","Type":"ContainerDied","Data":"09ade8beebc547b926c8ec338699ecc5081bd0a5abf6adb75752aa7513192671"}
Jul 22 12:10:19.591715 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.591227    2584 scope.go:117] "RemoveContainer" containerID="75e332697ac671d0ef8e99d22d4c19c79003fccf597337b601ddcfcc4045134a"
Jul 22 12:10:19.611472 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.611444    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"]
Jul 22 12:10:19.616759 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:19.616732    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mvbzj"]
Jul 22 12:10:21.525285 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:21.525251    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c50876e-87ce-4bfa-a1ab-cdb1fe80f921" path="/var/lib/kubelet/pods/9c50876e-87ce-4bfa-a1ab-cdb1fe80f921/volumes"
Jul 22 12:10:24.702715 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.702678    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"]
Jul 22 12:10:24.703192 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.703174    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c50876e-87ce-4bfa-a1ab-cdb1fe80f921" containerName="limitador"
Jul 22 12:10:24.703265 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.703196    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c50876e-87ce-4bfa-a1ab-cdb1fe80f921" containerName="limitador"
Jul 22 12:10:24.703317 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.703280    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c50876e-87ce-4bfa-a1ab-cdb1fe80f921" containerName="limitador"
Jul 22 12:10:24.706030 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.706008    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"
Jul 22 12:10:24.708882 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.708862    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:10:24.709011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.708885    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 22 12:10:24.709011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.708909    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-28k9d\""
Jul 22 12:10:24.710212 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.710199    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 12:10:24.719457 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.719436    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"]
Jul 22 12:10:24.764181 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.764143    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3ce581b-e7c0-41ad-8dcd-32f90b319e60-data\") pod \"postgres-69fb4594cf-m72j4\" (UID: \"f3ce581b-e7c0-41ad-8dcd-32f90b319e60\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"
Jul 22 12:10:24.764306 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.764209    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h27kn\" (UniqueName: \"kubernetes.io/projected/f3ce581b-e7c0-41ad-8dcd-32f90b319e60-kube-api-access-h27kn\") pod \"postgres-69fb4594cf-m72j4\" (UID: \"f3ce581b-e7c0-41ad-8dcd-32f90b319e60\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"
Jul 22 12:10:24.865203 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.865172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h27kn\" (UniqueName: \"kubernetes.io/projected/f3ce581b-e7c0-41ad-8dcd-32f90b319e60-kube-api-access-h27kn\") pod \"postgres-69fb4594cf-m72j4\" (UID: \"f3ce581b-e7c0-41ad-8dcd-32f90b319e60\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"
Jul 22 12:10:24.865348 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.865247    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3ce581b-e7c0-41ad-8dcd-32f90b319e60-data\") pod \"postgres-69fb4594cf-m72j4\" (UID: \"f3ce581b-e7c0-41ad-8dcd-32f90b319e60\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"
Jul 22 12:10:24.865581 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.865565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3ce581b-e7c0-41ad-8dcd-32f90b319e60-data\") pod \"postgres-69fb4594cf-m72j4\" (UID: \"f3ce581b-e7c0-41ad-8dcd-32f90b319e60\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"
Jul 22 12:10:24.875350 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:24.875334    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h27kn\" (UniqueName: \"kubernetes.io/projected/f3ce581b-e7c0-41ad-8dcd-32f90b319e60-kube-api-access-h27kn\") pod \"postgres-69fb4594cf-m72j4\" (UID: \"f3ce581b-e7c0-41ad-8dcd-32f90b319e60\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"
Jul 22 12:10:25.015722 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:25.015644    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"
Jul 22 12:10:25.173151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:25.173104    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"]
Jul 22 12:10:25.175652 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:10:25.175623    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3ce581b_e7c0_41ad_8dcd_32f90b319e60.slice/crio-49363f43eeea466e37adbf122905ec9c775577d3c2346bb060bd1ac478e6939c WatchSource:0}: Error finding container 49363f43eeea466e37adbf122905ec9c775577d3c2346bb060bd1ac478e6939c: Status 404 returned error can't find the container with id 49363f43eeea466e37adbf122905ec9c775577d3c2346bb060bd1ac478e6939c
Jul 22 12:10:25.618884 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:25.618852    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4" event={"ID":"f3ce581b-e7c0-41ad-8dcd-32f90b319e60","Type":"ContainerStarted","Data":"49363f43eeea466e37adbf122905ec9c775577d3c2346bb060bd1ac478e6939c"}
Jul 22 12:10:30.644043 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:30.644003    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4" event={"ID":"f3ce581b-e7c0-41ad-8dcd-32f90b319e60","Type":"ContainerStarted","Data":"09e105facf5e57162967d00300e9b901cd86faad339ddb7c9e799ab4b067695c"}
Jul 22 12:10:30.644419 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:30.644112    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"
Jul 22 12:10:30.663992 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:30.663941    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4" podStartSLOduration=1.5732782269999999 podStartE2EDuration="6.663921131s" totalImagesPullingTime="5.090642904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:10:30.661726091 +0000 UTC m=+519.683747808" watchObservedRunningTime="2026-07-22 12:10:30.663921131 +0000 UTC m=+519.685942833"
Jul 22 12:10:36.678881 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:36.678856    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m72j4"
Jul 22 12:10:44.927935 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:44.927858    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77bd68d6db-k56gl"]
Jul 22 12:10:44.930180 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:44.930158    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:10:44.935889 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:44.935869    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 12:10:44.936633 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:44.936521    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vtb92\""
Jul 22 12:10:44.937138 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:44.937120    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 12:10:44.951184 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:44.951164    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77bd68d6db-k56gl"]
Jul 22 12:10:45.057744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.057711    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8444ad4d-8278-4c72-8fb0-a48ad17f498b-webhook-cert\") pod \"maas-controller-77bd68d6db-k56gl\" (UID: \"8444ad4d-8278-4c72-8fb0-a48ad17f498b\") " pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:10:45.057925 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.057760    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwktk\" (UniqueName: \"kubernetes.io/projected/8444ad4d-8278-4c72-8fb0-a48ad17f498b-kube-api-access-pwktk\") pod \"maas-controller-77bd68d6db-k56gl\" (UID: \"8444ad4d-8278-4c72-8fb0-a48ad17f498b\") " pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:10:45.159046 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.159009    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8444ad4d-8278-4c72-8fb0-a48ad17f498b-webhook-cert\") pod \"maas-controller-77bd68d6db-k56gl\" (UID: \"8444ad4d-8278-4c72-8fb0-a48ad17f498b\") " pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:10:45.159233 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.159059    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwktk\" (UniqueName: \"kubernetes.io/projected/8444ad4d-8278-4c72-8fb0-a48ad17f498b-kube-api-access-pwktk\") pod \"maas-controller-77bd68d6db-k56gl\" (UID: \"8444ad4d-8278-4c72-8fb0-a48ad17f498b\") " pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:10:45.159233 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:10:45.159172    2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 22 12:10:45.159355 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:10:45.159243    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8444ad4d-8278-4c72-8fb0-a48ad17f498b-webhook-cert podName:8444ad4d-8278-4c72-8fb0-a48ad17f498b nodeName:}" failed. No retries permitted until 2026-07-22 12:10:45.659226502 +0000 UTC m=+534.681248181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/8444ad4d-8278-4c72-8fb0-a48ad17f498b-webhook-cert") pod "maas-controller-77bd68d6db-k56gl" (UID: "8444ad4d-8278-4c72-8fb0-a48ad17f498b") : secret "maas-controller-webhook-cert" not found
Jul 22 12:10:45.171730 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.171700    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwktk\" (UniqueName: \"kubernetes.io/projected/8444ad4d-8278-4c72-8fb0-a48ad17f498b-kube-api-access-pwktk\") pod \"maas-controller-77bd68d6db-k56gl\" (UID: \"8444ad4d-8278-4c72-8fb0-a48ad17f498b\") " pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:10:45.187257 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.187197    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58d867cfcf-p2nn4"]
Jul 22 12:10:45.190070 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.190051    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:10:45.204912 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.204885    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-p2nn4"]
Jul 22 12:10:45.361078 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.361045    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx9gf\" (UniqueName: \"kubernetes.io/projected/60e70333-d05f-4f7d-b550-3551dbb10f2b-kube-api-access-cx9gf\") pod \"maas-controller-58d867cfcf-p2nn4\" (UID: \"60e70333-d05f-4f7d-b550-3551dbb10f2b\") " pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:10:45.361247 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.361092    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60e70333-d05f-4f7d-b550-3551dbb10f2b-webhook-cert\") pod \"maas-controller-58d867cfcf-p2nn4\" (UID: \"60e70333-d05f-4f7d-b550-3551dbb10f2b\") " pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:10:45.462145 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.462048    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9gf\" (UniqueName: \"kubernetes.io/projected/60e70333-d05f-4f7d-b550-3551dbb10f2b-kube-api-access-cx9gf\") pod \"maas-controller-58d867cfcf-p2nn4\" (UID: \"60e70333-d05f-4f7d-b550-3551dbb10f2b\") " pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:10:45.462145 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.462104    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60e70333-d05f-4f7d-b550-3551dbb10f2b-webhook-cert\") pod \"maas-controller-58d867cfcf-p2nn4\" (UID: \"60e70333-d05f-4f7d-b550-3551dbb10f2b\") " pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:10:45.464632 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.464609    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60e70333-d05f-4f7d-b550-3551dbb10f2b-webhook-cert\") pod \"maas-controller-58d867cfcf-p2nn4\" (UID: \"60e70333-d05f-4f7d-b550-3551dbb10f2b\") " pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:10:45.478101 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.478075    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx9gf\" (UniqueName: \"kubernetes.io/projected/60e70333-d05f-4f7d-b550-3551dbb10f2b-kube-api-access-cx9gf\") pod \"maas-controller-58d867cfcf-p2nn4\" (UID: \"60e70333-d05f-4f7d-b550-3551dbb10f2b\") " pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:10:45.500030 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.499999    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:10:45.630454 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.630427    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-p2nn4"]
Jul 22 12:10:45.632018 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:10:45.631992    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e70333_d05f_4f7d_b550_3551dbb10f2b.slice/crio-0e86b1cc8b8c74ec195cd98820ea23c72b8cab49f9aed2f52127536ed00e4599 WatchSource:0}: Error finding container 0e86b1cc8b8c74ec195cd98820ea23c72b8cab49f9aed2f52127536ed00e4599: Status 404 returned error can't find the container with id 0e86b1cc8b8c74ec195cd98820ea23c72b8cab49f9aed2f52127536ed00e4599
Jul 22 12:10:45.664286 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.664253    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8444ad4d-8278-4c72-8fb0-a48ad17f498b-webhook-cert\") pod \"maas-controller-77bd68d6db-k56gl\" (UID: \"8444ad4d-8278-4c72-8fb0-a48ad17f498b\") " pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:10:45.666567 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.666546    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8444ad4d-8278-4c72-8fb0-a48ad17f498b-webhook-cert\") pod \"maas-controller-77bd68d6db-k56gl\" (UID: \"8444ad4d-8278-4c72-8fb0-a48ad17f498b\") " pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:10:45.717168 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.717096    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d867cfcf-p2nn4" event={"ID":"60e70333-d05f-4f7d-b550-3551dbb10f2b","Type":"ContainerStarted","Data":"0e86b1cc8b8c74ec195cd98820ea23c72b8cab49f9aed2f52127536ed00e4599"}
Jul 22 12:10:45.839890 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.839861    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:10:45.968335 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:45.968309    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77bd68d6db-k56gl"]
Jul 22 12:10:45.969926 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:10:45.969894    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8444ad4d_8278_4c72_8fb0_a48ad17f498b.slice/crio-e55bc7df35a8e525d1bfb963513b825805c6066edf4184c1a5f8a211d9c0b468 WatchSource:0}: Error finding container e55bc7df35a8e525d1bfb963513b825805c6066edf4184c1a5f8a211d9c0b468: Status 404 returned error can't find the container with id e55bc7df35a8e525d1bfb963513b825805c6066edf4184c1a5f8a211d9c0b468
Jul 22 12:10:46.729910 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:46.729868    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77bd68d6db-k56gl" event={"ID":"8444ad4d-8278-4c72-8fb0-a48ad17f498b","Type":"ContainerStarted","Data":"e55bc7df35a8e525d1bfb963513b825805c6066edf4184c1a5f8a211d9c0b468"}
Jul 22 12:10:48.742697 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:48.742661    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d867cfcf-p2nn4" event={"ID":"60e70333-d05f-4f7d-b550-3551dbb10f2b","Type":"ContainerStarted","Data":"fc5aade24728ff03919d818f3e9787f86574748aa31465c8c59f39bd7cc54965"}
Jul 22 12:10:48.743174 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:48.742770    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:10:48.744449 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:48.744412    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77bd68d6db-k56gl" event={"ID":"8444ad4d-8278-4c72-8fb0-a48ad17f498b","Type":"ContainerStarted","Data":"9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815"}
Jul 22 12:10:48.744573 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:48.744493    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:10:48.805490 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:48.805433    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58d867cfcf-p2nn4" podStartSLOduration=1.248642789 podStartE2EDuration="3.80536513s" totalImagesPullingTime="2.556722341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:10:48.802269907 +0000 UTC m=+537.824291648" watchObservedRunningTime="2026-07-22 12:10:48.80536513 +0000 UTC m=+537.827386832"
Jul 22 12:10:59.751895 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:59.751863    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:10:59.752255 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:59.752047    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:10:59.773065 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:59.773016    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77bd68d6db-k56gl" podStartSLOduration=13.548119655 podStartE2EDuration="15.773001734s" totalImagesPullingTime="2.224882079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:10:48.847582958 +0000 UTC m=+537.869604678" watchObservedRunningTime="2026-07-22 12:10:59.773001734 +0000 UTC m=+548.795023435"
Jul 22 12:10:59.822820 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:59.822782    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77bd68d6db-k56gl"]
Jul 22 12:10:59.823030 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:10:59.822983    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77bd68d6db-k56gl" podUID="8444ad4d-8278-4c72-8fb0-a48ad17f498b" containerName="manager" containerID="cri-o://9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815" gracePeriod=30
Jul 22 12:11:00.078344 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.078319    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:11:00.187304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.187272    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8444ad4d-8278-4c72-8fb0-a48ad17f498b-webhook-cert\") pod \"8444ad4d-8278-4c72-8fb0-a48ad17f498b\" (UID: \"8444ad4d-8278-4c72-8fb0-a48ad17f498b\") "
Jul 22 12:11:00.187304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.187311    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwktk\" (UniqueName: \"kubernetes.io/projected/8444ad4d-8278-4c72-8fb0-a48ad17f498b-kube-api-access-pwktk\") pod \"8444ad4d-8278-4c72-8fb0-a48ad17f498b\" (UID: \"8444ad4d-8278-4c72-8fb0-a48ad17f498b\") "
Jul 22 12:11:00.189444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.189415    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8444ad4d-8278-4c72-8fb0-a48ad17f498b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8444ad4d-8278-4c72-8fb0-a48ad17f498b" (UID: "8444ad4d-8278-4c72-8fb0-a48ad17f498b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:11:00.189444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.189437    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8444ad4d-8278-4c72-8fb0-a48ad17f498b-kube-api-access-pwktk" (OuterVolumeSpecName: "kube-api-access-pwktk") pod "8444ad4d-8278-4c72-8fb0-a48ad17f498b" (UID: "8444ad4d-8278-4c72-8fb0-a48ad17f498b"). InnerVolumeSpecName "kube-api-access-pwktk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:11:00.288975 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.288938    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8444ad4d-8278-4c72-8fb0-a48ad17f498b-webhook-cert\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:11:00.288975 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.288970    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwktk\" (UniqueName: \"kubernetes.io/projected/8444ad4d-8278-4c72-8fb0-a48ad17f498b-kube-api-access-pwktk\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:11:00.800638 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.800607    2584 generic.go:356] "Generic (PLEG): container finished" podID="8444ad4d-8278-4c72-8fb0-a48ad17f498b" containerID="9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815" exitCode=0
Jul 22 12:11:00.801087 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.800646    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77bd68d6db-k56gl"
Jul 22 12:11:00.801087 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.800696    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77bd68d6db-k56gl" event={"ID":"8444ad4d-8278-4c72-8fb0-a48ad17f498b","Type":"ContainerDied","Data":"9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815"}
Jul 22 12:11:00.801087 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.800758    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77bd68d6db-k56gl" event={"ID":"8444ad4d-8278-4c72-8fb0-a48ad17f498b","Type":"ContainerDied","Data":"e55bc7df35a8e525d1bfb963513b825805c6066edf4184c1a5f8a211d9c0b468"}
Jul 22 12:11:00.801087 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.800793    2584 scope.go:117] "RemoveContainer" containerID="9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815"
Jul 22 12:11:00.810424 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.810406    2584 scope.go:117] "RemoveContainer" containerID="9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815"
Jul 22 12:11:00.810695 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:11:00.810677    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815\": container with ID starting with 9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815 not found: ID does not exist" containerID="9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815"
Jul 22 12:11:00.810744 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.810704    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815"} err="failed to get container status \"9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815\": rpc error: code = NotFound desc = could not find container \"9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815\": container with ID starting with 9d24e4736a82ddd533e8765b28c217af2f0678bd875e0d0f70a90822cb35c815 not found: ID does not exist"
Jul 22 12:11:00.833014 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.832989    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77bd68d6db-k56gl"]
Jul 22 12:11:00.834418 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:00.834400    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77bd68d6db-k56gl"]
Jul 22 12:11:01.524308 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:01.524276    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8444ad4d-8278-4c72-8fb0-a48ad17f498b" path="/var/lib/kubelet/pods/8444ad4d-8278-4c72-8fb0-a48ad17f498b/volumes"
Jul 22 12:11:19.802200 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:19.802165    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"]
Jul 22 12:11:19.802665 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:19.802527    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8444ad4d-8278-4c72-8fb0-a48ad17f498b" containerName="manager"
Jul 22 12:11:19.802665 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:19.802539    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8444ad4d-8278-4c72-8fb0-a48ad17f498b" containerName="manager"
Jul 22 12:11:19.802665 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:19.802599    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8444ad4d-8278-4c72-8fb0-a48ad17f498b" containerName="manager"
Jul 22 12:11:19.807648 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:19.807628    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:19.810885 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:19.810859    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 22 12:11:19.818765 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:19.818741    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"]
Jul 22 12:11:19.958873 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:19.958841    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f91fa26b-8859-4de9-a73d-48f2550d32e6-maas-api-tls\") pod \"maas-api-7bd5fc67c4-x6rwt\" (UID: \"f91fa26b-8859-4de9-a73d-48f2550d32e6\") " pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:19.958873 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:19.958876    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f91fa26b-8859-4de9-a73d-48f2550d32e6-openshift-service-ca\") pod \"maas-api-7bd5fc67c4-x6rwt\" (UID: \"f91fa26b-8859-4de9-a73d-48f2550d32e6\") " pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:19.959076 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:19.958903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4tzg\" (UniqueName: \"kubernetes.io/projected/f91fa26b-8859-4de9-a73d-48f2550d32e6-kube-api-access-w4tzg\") pod \"maas-api-7bd5fc67c4-x6rwt\" (UID: \"f91fa26b-8859-4de9-a73d-48f2550d32e6\") " pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:20.059738 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:20.059660    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f91fa26b-8859-4de9-a73d-48f2550d32e6-maas-api-tls\") pod \"maas-api-7bd5fc67c4-x6rwt\" (UID: \"f91fa26b-8859-4de9-a73d-48f2550d32e6\") " pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:20.059738 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:20.059694    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f91fa26b-8859-4de9-a73d-48f2550d32e6-openshift-service-ca\") pod \"maas-api-7bd5fc67c4-x6rwt\" (UID: \"f91fa26b-8859-4de9-a73d-48f2550d32e6\") " pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:20.059738 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:20.059722    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4tzg\" (UniqueName: \"kubernetes.io/projected/f91fa26b-8859-4de9-a73d-48f2550d32e6-kube-api-access-w4tzg\") pod \"maas-api-7bd5fc67c4-x6rwt\" (UID: \"f91fa26b-8859-4de9-a73d-48f2550d32e6\") " pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:20.060428 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:20.060408    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f91fa26b-8859-4de9-a73d-48f2550d32e6-openshift-service-ca\") pod \"maas-api-7bd5fc67c4-x6rwt\" (UID: \"f91fa26b-8859-4de9-a73d-48f2550d32e6\") " pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:20.062209 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:20.062186    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f91fa26b-8859-4de9-a73d-48f2550d32e6-maas-api-tls\") pod \"maas-api-7bd5fc67c4-x6rwt\" (UID: \"f91fa26b-8859-4de9-a73d-48f2550d32e6\") " pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:20.071911 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:20.071889    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4tzg\" (UniqueName: \"kubernetes.io/projected/f91fa26b-8859-4de9-a73d-48f2550d32e6-kube-api-access-w4tzg\") pod \"maas-api-7bd5fc67c4-x6rwt\" (UID: \"f91fa26b-8859-4de9-a73d-48f2550d32e6\") " pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:20.117412 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:20.117368    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:20.256677 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:20.256539    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"]
Jul 22 12:11:20.259526 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:11:20.259500    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf91fa26b_8859_4de9_a73d_48f2550d32e6.slice/crio-bde5e2979e6bd58e741a9563e2641797345bec662ff5fe3c0ac750db0f0d08ad WatchSource:0}: Error finding container bde5e2979e6bd58e741a9563e2641797345bec662ff5fe3c0ac750db0f0d08ad: Status 404 returned error can't find the container with id bde5e2979e6bd58e741a9563e2641797345bec662ff5fe3c0ac750db0f0d08ad
Jul 22 12:11:20.886735 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:20.886702    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt" event={"ID":"f91fa26b-8859-4de9-a73d-48f2550d32e6","Type":"ContainerStarted","Data":"bde5e2979e6bd58e741a9563e2641797345bec662ff5fe3c0ac750db0f0d08ad"}
Jul 22 12:11:22.897680 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:22.897644    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt" event={"ID":"f91fa26b-8859-4de9-a73d-48f2550d32e6","Type":"ContainerStarted","Data":"f3589088b4d7aee929fefb4ed9487dfd10a3d31b84744a923d14790ae79b2080"}
Jul 22 12:11:22.898161 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:22.897773    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:22.919445 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:22.919402    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt" podStartSLOduration=1.916755188 podStartE2EDuration="3.919372152s" totalImagesPullingTime="2.002616964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:11:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:11:22.917434592 +0000 UTC m=+571.939456292" watchObservedRunningTime="2026-07-22 12:11:22.919372152 +0000 UTC m=+571.941393854"
Jul 22 12:11:28.908273 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:28.908244    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7bd5fc67c4-x6rwt"
Jul 22 12:11:28.927065 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:28.927034    2584 generic.go:356] "Generic (PLEG): container finished" podID="e5c60d29-9b5e-4b24-a0ac-4e8d118c2270" containerID="6c0962cb00db2429bc57fd7577df1dba1c2b46b5dcdb1b76694970e297e24107" exitCode=0
Jul 22 12:11:28.927208 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:28.927102    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks" event={"ID":"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270","Type":"ContainerDied","Data":"6c0962cb00db2429bc57fd7577df1dba1c2b46b5dcdb1b76694970e297e24107"}
Jul 22 12:11:28.927503 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:28.927489    2584 scope.go:117] "RemoveContainer" containerID="6c0962cb00db2429bc57fd7577df1dba1c2b46b5dcdb1b76694970e297e24107"
Jul 22 12:11:28.928507 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:28.928471    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:11:28.928618 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:28.928547    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:11:28.928618 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:28.928587    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:11:29.934168 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:29.934136    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks" event={"ID":"e5c60d29-9b5e-4b24-a0ac-4e8d118c2270","Type":"ContainerStarted","Data":"174660dc83d934fa100e076cde3186f8e40f29c5a2c3d134266cc708de690609"}
Jul 22 12:11:30.349058 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:30.349028    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:11:30.350400 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:30.350353    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fhvks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:11:30.350538 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:30.350428    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks" podUID="e5c60d29-9b5e-4b24-a0ac-4e8d118c2270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 22 12:11:31.349034 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:31.348974    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fhvks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:11:31.349542 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:31.349049    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks" podUID="e5c60d29-9b5e-4b24-a0ac-4e8d118c2270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 22 12:11:32.349142 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:32.349098    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fhvks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:11:32.349708 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:32.349197    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks" podUID="e5c60d29-9b5e-4b24-a0ac-4e8d118c2270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 22 12:11:32.510275 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:32.510237    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:11:33.349310 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:33.349269    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fhvks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:11:33.349943 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:33.349906    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks" podUID="e5c60d29-9b5e-4b24-a0ac-4e8d118c2270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 22 12:11:34.348989 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:34.348944    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fhvks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:11:34.349194 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:34.349013    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks" podUID="e5c60d29-9b5e-4b24-a0ac-4e8d118c2270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 22 12:11:35.354102 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:35.354072    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:11:35.355034 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:11:35.355011    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fhvks"
Jul 22 12:12:02.192118 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.192086    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"]
Jul 22 12:12:02.195368 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.195349    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.198615 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.198596    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 12:12:02.198728 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.198622    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 12:12:02.198882 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.198867    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 22 12:12:02.198926 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.198871    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gf6j9\""
Jul 22 12:12:02.209305 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.209282    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"]
Jul 22 12:12:02.343542 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.343512    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.343542 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.343543    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnxs7\" (UniqueName: \"kubernetes.io/projected/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-kube-api-access-jnxs7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.343740 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.343646    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.343740 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.343711    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.343811 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.343784    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.343845 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.343807    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.444558 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.444475    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.444558 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.444538    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.444821 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.444574    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.444821 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.444590    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.444821 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.444620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.444821 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.444636    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnxs7\" (UniqueName: \"kubernetes.io/projected/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-kube-api-access-jnxs7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.445034 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.444928    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.445034 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.444979    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.445034 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.444988    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.446853 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.446829    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.446962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.446945    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.458212 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.458191    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnxs7\" (UniqueName: \"kubernetes.io/projected/d4d00e16-c225-4ed0-b8ba-139e4eca75f8-kube-api-access-jnxs7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5\" (UID: \"d4d00e16-c225-4ed0-b8ba-139e4eca75f8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.505950 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.505908    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:02.643917 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:02.643893    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"]
Jul 22 12:12:02.645001 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:12:02.644981    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d00e16_c225_4ed0_b8ba_139e4eca75f8.slice/crio-d23594047dd488ef8a5bcc744d9191ae5c3c693423667cb89eadc6156b80f3a4 WatchSource:0}: Error finding container d23594047dd488ef8a5bcc744d9191ae5c3c693423667cb89eadc6156b80f3a4: Status 404 returned error can't find the container with id d23594047dd488ef8a5bcc744d9191ae5c3c693423667cb89eadc6156b80f3a4
Jul 22 12:12:03.096254 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:03.096221    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5" event={"ID":"d4d00e16-c225-4ed0-b8ba-139e4eca75f8","Type":"ContainerStarted","Data":"d23594047dd488ef8a5bcc744d9191ae5c3c693423667cb89eadc6156b80f3a4"}
Jul 22 12:12:05.106050 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:05.106017    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5" event={"ID":"d4d00e16-c225-4ed0-b8ba-139e4eca75f8","Type":"ContainerStarted","Data":"225293cf2a452df223efcd631864cdff4b4ffd3fa7e6ffce217a02accbc4cf79"}
Jul 22 12:12:05.133992 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:05.133942    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5" podStartSLOduration=1.620657295 podStartE2EDuration="3.133928673s" totalImagesPullingTime="1.513271378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:12:05.13342453 +0000 UTC m=+614.155446230" watchObservedRunningTime="2026-07-22 12:12:05.133928673 +0000 UTC m=+614.155950373"
Jul 22 12:12:12.506306 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:12.506271    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:12.506729 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:12.506353    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:12.518575 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:12.518553    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:13.153684 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:13.153654    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5"
Jul 22 12:12:26.001076 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.001044    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"]
Jul 22 12:12:26.003763 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.003739    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.006819 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.006754    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 22 12:12:26.019535 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.019509    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"]
Jul 22 12:12:26.045290 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.045268    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.045440 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.045301    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.045440 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.045318    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.045440 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.045333    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.045558 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.045449    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.045558 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.045488    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dqlf\" (UniqueName: \"kubernetes.io/projected/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-kube-api-access-5dqlf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.146228 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.146195    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.146448 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.146244    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dqlf\" (UniqueName: \"kubernetes.io/projected/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-kube-api-access-5dqlf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.146448 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.146313    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.146448 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.146348    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.146448 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.146372    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.146448 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.146424    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.146787 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.146583    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.146787 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.146719    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.146880 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.146823    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.148649 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.148622    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.148799 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.148783    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.156251 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.156230    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dqlf\" (UniqueName: \"kubernetes.io/projected/ea714d9e-fd4a-42f5-8f5d-1d2d4406a859-kube-api-access-5dqlf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-56hxt\" (UID: \"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.315597 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.315526    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:26.451454 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.451431    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"]
Jul 22 12:12:26.454153 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:12:26.454125    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea714d9e_fd4a_42f5_8f5d_1d2d4406a859.slice/crio-59560b517770c5e02a763e5c0a87a9db42417cf9fb3df1a68772580a48c642ea WatchSource:0}: Error finding container 59560b517770c5e02a763e5c0a87a9db42417cf9fb3df1a68772580a48c642ea: Status 404 returned error can't find the container with id 59560b517770c5e02a763e5c0a87a9db42417cf9fb3df1a68772580a48c642ea
Jul 22 12:12:26.455525 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:26.455508    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:12:27.207401 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:27.207346    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt" event={"ID":"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859","Type":"ContainerStarted","Data":"4d925007d9f730b63697eb594e3587acc7c737a392c74034f3a5409f3b5e0747"}
Jul 22 12:12:27.207847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:27.207417    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt" event={"ID":"ea714d9e-fd4a-42f5-8f5d-1d2d4406a859","Type":"ContainerStarted","Data":"59560b517770c5e02a763e5c0a87a9db42417cf9fb3df1a68772580a48c642ea"}
Jul 22 12:12:36.315934 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:36.315898    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:36.316450 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:36.315958    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:36.328466 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:36.328443    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:36.349687 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:36.349642    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt" podStartSLOduration=11.151689946 podStartE2EDuration="11.349628718s" totalImagesPullingTime="197.938772ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:12:27.243479346 +0000 UTC m=+636.265501050" watchObservedRunningTime="2026-07-22 12:12:36.349628718 +0000 UTC m=+645.371650418"
Jul 22 12:12:37.261612 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:37.261575    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-56hxt"
Jul 22 12:12:44.679641 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.679601    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"]
Jul 22 12:12:44.685260 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.685231    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.690297 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.690270    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 22 12:12:44.697051 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.697026    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"]
Jul 22 12:12:44.810175 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.810140    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0801a305-ec36-4442-91a1-43a99ef3ad7a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.810358 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.810219    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.810358 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.810241    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.810358 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.810282    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.810358 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.810307    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5mps\" (UniqueName: \"kubernetes.io/projected/0801a305-ec36-4442-91a1-43a99ef3ad7a-kube-api-access-q5mps\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.810552 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.810412    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.911247 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.911215    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.911247 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.911249    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.911475 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.911285    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.911475 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.911303    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5mps\" (UniqueName: \"kubernetes.io/projected/0801a305-ec36-4442-91a1-43a99ef3ad7a-kube-api-access-q5mps\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.911475 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.911422    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.911475 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.911463    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0801a305-ec36-4442-91a1-43a99ef3ad7a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.911689 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.911663    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.911746 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.911706    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.911746 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.911721    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.913632 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.913608    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0801a305-ec36-4442-91a1-43a99ef3ad7a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.913861 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.913840    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0801a305-ec36-4442-91a1-43a99ef3ad7a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.920610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.920590    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5mps\" (UniqueName: \"kubernetes.io/projected/0801a305-ec36-4442-91a1-43a99ef3ad7a-kube-api-access-q5mps\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scvlf\" (UID: \"0801a305-ec36-4442-91a1-43a99ef3ad7a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:44.997644 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:44.997554    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:45.135815 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:45.135792    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"]
Jul 22 12:12:45.138725 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:12:45.138695    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0801a305_ec36_4442_91a1_43a99ef3ad7a.slice/crio-a2c6cb07d828cb1b4dd44da0fda2ee039f81f92c46d1278007548d8607466b54 WatchSource:0}: Error finding container a2c6cb07d828cb1b4dd44da0fda2ee039f81f92c46d1278007548d8607466b54: Status 404 returned error can't find the container with id a2c6cb07d828cb1b4dd44da0fda2ee039f81f92c46d1278007548d8607466b54
Jul 22 12:12:45.286766 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:45.286682    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf" event={"ID":"0801a305-ec36-4442-91a1-43a99ef3ad7a","Type":"ContainerStarted","Data":"a2c6cb07d828cb1b4dd44da0fda2ee039f81f92c46d1278007548d8607466b54"}
Jul 22 12:12:46.294035 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:46.294006    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf" event={"ID":"0801a305-ec36-4442-91a1-43a99ef3ad7a","Type":"ContainerStarted","Data":"64e915575c5585d307c0a4b437a02a61198541b2134a7ed12c4fa2eca135b159"}
Jul 22 12:12:46.320229 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:46.320173    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf" podStartSLOduration=1.9536212069999999 podStartE2EDuration="2.320157617s" totalImagesPullingTime="366.53641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:12:46.318427683 +0000 UTC m=+655.340449383" watchObservedRunningTime="2026-07-22 12:12:46.320157617 +0000 UTC m=+655.342179317"
Jul 22 12:12:54.998508 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:54.998453    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:54.998999 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:54.998554    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:55.011245 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:55.011213    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:12:55.352329 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:12:55.352302    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scvlf"
Jul 22 12:14:47.607699 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:47.607664    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-p2nn4"]
Jul 22 12:14:47.608296 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:47.607865    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-58d867cfcf-p2nn4" podUID="60e70333-d05f-4f7d-b550-3551dbb10f2b" containerName="manager" containerID="cri-o://fc5aade24728ff03919d818f3e9787f86574748aa31465c8c59f39bd7cc54965" gracePeriod=30
Jul 22 12:14:47.859144 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:47.859081    2584 generic.go:356] "Generic (PLEG): container finished" podID="60e70333-d05f-4f7d-b550-3551dbb10f2b" containerID="fc5aade24728ff03919d818f3e9787f86574748aa31465c8c59f39bd7cc54965" exitCode=0
Jul 22 12:14:47.859254 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:47.859154    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d867cfcf-p2nn4" event={"ID":"60e70333-d05f-4f7d-b550-3551dbb10f2b","Type":"ContainerDied","Data":"fc5aade24728ff03919d818f3e9787f86574748aa31465c8c59f39bd7cc54965"}
Jul 22 12:14:47.859254 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:47.859206    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d867cfcf-p2nn4" event={"ID":"60e70333-d05f-4f7d-b550-3551dbb10f2b","Type":"ContainerDied","Data":"0e86b1cc8b8c74ec195cd98820ea23c72b8cab49f9aed2f52127536ed00e4599"}
Jul 22 12:14:47.859254 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:47.859223    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e86b1cc8b8c74ec195cd98820ea23c72b8cab49f9aed2f52127536ed00e4599"
Jul 22 12:14:47.860484 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:47.860468    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:14:47.930908 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:47.930877    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60e70333-d05f-4f7d-b550-3551dbb10f2b-webhook-cert\") pod \"60e70333-d05f-4f7d-b550-3551dbb10f2b\" (UID: \"60e70333-d05f-4f7d-b550-3551dbb10f2b\") "
Jul 22 12:14:47.931086 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:47.931000    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx9gf\" (UniqueName: \"kubernetes.io/projected/60e70333-d05f-4f7d-b550-3551dbb10f2b-kube-api-access-cx9gf\") pod \"60e70333-d05f-4f7d-b550-3551dbb10f2b\" (UID: \"60e70333-d05f-4f7d-b550-3551dbb10f2b\") "
Jul 22 12:14:47.933028 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:47.932996    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e70333-d05f-4f7d-b550-3551dbb10f2b-kube-api-access-cx9gf" (OuterVolumeSpecName: "kube-api-access-cx9gf") pod "60e70333-d05f-4f7d-b550-3551dbb10f2b" (UID: "60e70333-d05f-4f7d-b550-3551dbb10f2b"). InnerVolumeSpecName "kube-api-access-cx9gf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:14:47.933148 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:47.933089    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e70333-d05f-4f7d-b550-3551dbb10f2b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "60e70333-d05f-4f7d-b550-3551dbb10f2b" (UID: "60e70333-d05f-4f7d-b550-3551dbb10f2b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:14:48.031803 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:48.031771    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx9gf\" (UniqueName: \"kubernetes.io/projected/60e70333-d05f-4f7d-b550-3551dbb10f2b-kube-api-access-cx9gf\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:14:48.031803 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:48.031797    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60e70333-d05f-4f7d-b550-3551dbb10f2b-webhook-cert\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:14:48.864038 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:48.864006    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d867cfcf-p2nn4"
Jul 22 12:14:48.900151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:48.900119    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-p2nn4"]
Jul 22 12:14:48.906861 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:48.906834    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-p2nn4"]
Jul 22 12:14:49.525557 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:14:49.525519    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60e70333-d05f-4f7d-b550-3551dbb10f2b" path="/var/lib/kubelet/pods/60e70333-d05f-4f7d-b550-3551dbb10f2b/volumes"
Jul 22 12:15:49.762028 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.761997    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58d867cfcf-hmk4k"]
Jul 22 12:15:49.762497 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.762360    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e70333-d05f-4f7d-b550-3551dbb10f2b" containerName="manager"
Jul 22 12:15:49.762497 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.762372    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e70333-d05f-4f7d-b550-3551dbb10f2b" containerName="manager"
Jul 22 12:15:49.762497 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.762448    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e70333-d05f-4f7d-b550-3551dbb10f2b" containerName="manager"
Jul 22 12:15:49.764410 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.764373    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:15:49.767477 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.767456    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 12:15:49.768963 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.768935    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 12:15:49.768963 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.768943    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vtb92\""
Jul 22 12:15:49.775648 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.775629    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-hmk4k"]
Jul 22 12:15:49.895669 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.895632    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sqz9\" (UniqueName: \"kubernetes.io/projected/41ecb038-32cf-406d-8dca-f27914c5e9b8-kube-api-access-7sqz9\") pod \"maas-controller-58d867cfcf-hmk4k\" (UID: \"41ecb038-32cf-406d-8dca-f27914c5e9b8\") " pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:15:49.895669 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.895671    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41ecb038-32cf-406d-8dca-f27914c5e9b8-webhook-cert\") pod \"maas-controller-58d867cfcf-hmk4k\" (UID: \"41ecb038-32cf-406d-8dca-f27914c5e9b8\") " pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:15:49.996090 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.996054    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sqz9\" (UniqueName: \"kubernetes.io/projected/41ecb038-32cf-406d-8dca-f27914c5e9b8-kube-api-access-7sqz9\") pod \"maas-controller-58d867cfcf-hmk4k\" (UID: \"41ecb038-32cf-406d-8dca-f27914c5e9b8\") " pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:15:49.996090 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.996096    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41ecb038-32cf-406d-8dca-f27914c5e9b8-webhook-cert\") pod \"maas-controller-58d867cfcf-hmk4k\" (UID: \"41ecb038-32cf-406d-8dca-f27914c5e9b8\") " pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:15:49.998533 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:49.998517    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41ecb038-32cf-406d-8dca-f27914c5e9b8-webhook-cert\") pod \"maas-controller-58d867cfcf-hmk4k\" (UID: \"41ecb038-32cf-406d-8dca-f27914c5e9b8\") " pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:15:50.006559 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:50.006535    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sqz9\" (UniqueName: \"kubernetes.io/projected/41ecb038-32cf-406d-8dca-f27914c5e9b8-kube-api-access-7sqz9\") pod \"maas-controller-58d867cfcf-hmk4k\" (UID: \"41ecb038-32cf-406d-8dca-f27914c5e9b8\") " pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:15:50.074396 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:50.074316    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:15:50.211694 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:50.211671    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-hmk4k"]
Jul 22 12:15:50.213793 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:15:50.213772    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ecb038_32cf_406d_8dca_f27914c5e9b8.slice/crio-1ecd51a207892a55af60f8c1eb2c5bcac1341098815b1fab5776f778defc48f4 WatchSource:0}: Error finding container 1ecd51a207892a55af60f8c1eb2c5bcac1341098815b1fab5776f778defc48f4: Status 404 returned error can't find the container with id 1ecd51a207892a55af60f8c1eb2c5bcac1341098815b1fab5776f778defc48f4
Jul 22 12:15:51.152277 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:51.152243    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d867cfcf-hmk4k" event={"ID":"41ecb038-32cf-406d-8dca-f27914c5e9b8","Type":"ContainerStarted","Data":"f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699"}
Jul 22 12:15:51.152778 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:51.152306    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d867cfcf-hmk4k" event={"ID":"41ecb038-32cf-406d-8dca-f27914c5e9b8","Type":"ContainerStarted","Data":"1ecd51a207892a55af60f8c1eb2c5bcac1341098815b1fab5776f778defc48f4"}
Jul 22 12:15:51.152778 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:51.152335    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:15:51.174667 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:15:51.174617    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58d867cfcf-hmk4k" podStartSLOduration=1.586050784 podStartE2EDuration="2.174599851s" totalImagesPullingTime="588.549067ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:15:51.172975374 +0000 UTC m=+840.194997076" watchObservedRunningTime="2026-07-22 12:15:51.174599851 +0000 UTC m=+840.196621581"
Jul 22 12:16:02.159212 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:02.159179    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:16:06.891049 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:06.891017    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-hmk4k"]
Jul 22 12:16:06.891461 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:06.891210    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-58d867cfcf-hmk4k" podUID="41ecb038-32cf-406d-8dca-f27914c5e9b8" containerName="manager" containerID="cri-o://f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699" gracePeriod=30
Jul 22 12:16:07.142003 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.141946    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:16:07.143943 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.143923    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sqz9\" (UniqueName: \"kubernetes.io/projected/41ecb038-32cf-406d-8dca-f27914c5e9b8-kube-api-access-7sqz9\") pod \"41ecb038-32cf-406d-8dca-f27914c5e9b8\" (UID: \"41ecb038-32cf-406d-8dca-f27914c5e9b8\") "
Jul 22 12:16:07.144009 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.143994    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41ecb038-32cf-406d-8dca-f27914c5e9b8-webhook-cert\") pod \"41ecb038-32cf-406d-8dca-f27914c5e9b8\" (UID: \"41ecb038-32cf-406d-8dca-f27914c5e9b8\") "
Jul 22 12:16:07.145924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.145895    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ecb038-32cf-406d-8dca-f27914c5e9b8-kube-api-access-7sqz9" (OuterVolumeSpecName: "kube-api-access-7sqz9") pod "41ecb038-32cf-406d-8dca-f27914c5e9b8" (UID: "41ecb038-32cf-406d-8dca-f27914c5e9b8"). InnerVolumeSpecName "kube-api-access-7sqz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:16:07.146129 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.146107    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41ecb038-32cf-406d-8dca-f27914c5e9b8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "41ecb038-32cf-406d-8dca-f27914c5e9b8" (UID: "41ecb038-32cf-406d-8dca-f27914c5e9b8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:16:07.230051 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.230020    2584 generic.go:356] "Generic (PLEG): container finished" podID="41ecb038-32cf-406d-8dca-f27914c5e9b8" containerID="f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699" exitCode=0
Jul 22 12:16:07.230051 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.230055    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d867cfcf-hmk4k" event={"ID":"41ecb038-32cf-406d-8dca-f27914c5e9b8","Type":"ContainerDied","Data":"f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699"}
Jul 22 12:16:07.230311 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.230067    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d867cfcf-hmk4k"
Jul 22 12:16:07.230311 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.230096    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d867cfcf-hmk4k" event={"ID":"41ecb038-32cf-406d-8dca-f27914c5e9b8","Type":"ContainerDied","Data":"1ecd51a207892a55af60f8c1eb2c5bcac1341098815b1fab5776f778defc48f4"}
Jul 22 12:16:07.230311 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.230117    2584 scope.go:117] "RemoveContainer" containerID="f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699"
Jul 22 12:16:07.239837 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.239817    2584 scope.go:117] "RemoveContainer" containerID="f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699"
Jul 22 12:16:07.240123 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:16:07.240103    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699\": container with ID starting with f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699 not found: ID does not exist" containerID="f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699"
Jul 22 12:16:07.240202 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.240130    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699"} err="failed to get container status \"f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699\": rpc error: code = NotFound desc = could not find container \"f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699\": container with ID starting with f8d09f0b50c34840842b7a386bcc6ccd9526eefcd594f21e40ed732e28ff5699 not found: ID does not exist"
Jul 22 12:16:07.244862 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.244839    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sqz9\" (UniqueName: \"kubernetes.io/projected/41ecb038-32cf-406d-8dca-f27914c5e9b8-kube-api-access-7sqz9\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:16:07.244962 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.244866    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41ecb038-32cf-406d-8dca-f27914c5e9b8-webhook-cert\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:16:07.257950 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.257931    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-hmk4k"]
Jul 22 12:16:07.267713 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.267692    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-hmk4k"]
Jul 22 12:16:07.525183 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:07.525110    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41ecb038-32cf-406d-8dca-f27914c5e9b8" path="/var/lib/kubelet/pods/41ecb038-32cf-406d-8dca-f27914c5e9b8/volumes"
Jul 22 12:16:08.158442 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.158410    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58d867cfcf-zxv9g"]
Jul 22 12:16:08.158901 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.158781    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ecb038-32cf-406d-8dca-f27914c5e9b8" containerName="manager"
Jul 22 12:16:08.158901 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.158818    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ecb038-32cf-406d-8dca-f27914c5e9b8" containerName="manager"
Jul 22 12:16:08.158901 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.158900    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="41ecb038-32cf-406d-8dca-f27914c5e9b8" containerName="manager"
Jul 22 12:16:08.162163 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.162140    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d867cfcf-zxv9g"
Jul 22 12:16:08.165426 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.165405    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vtb92\""
Jul 22 12:16:08.166430 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.166411    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 12:16:08.166571 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.166412    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 12:16:08.180717 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.180690    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-zxv9g"]
Jul 22 12:16:08.250775 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.250749    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp4pr\" (UniqueName: \"kubernetes.io/projected/8b94f66f-f022-414b-8784-603142645fde-kube-api-access-pp4pr\") pod \"maas-controller-58d867cfcf-zxv9g\" (UID: \"8b94f66f-f022-414b-8784-603142645fde\") " pod="opendatahub/maas-controller-58d867cfcf-zxv9g"
Jul 22 12:16:08.250945 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.250791    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b94f66f-f022-414b-8784-603142645fde-webhook-cert\") pod \"maas-controller-58d867cfcf-zxv9g\" (UID: \"8b94f66f-f022-414b-8784-603142645fde\") " pod="opendatahub/maas-controller-58d867cfcf-zxv9g"
Jul 22 12:16:08.352135 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.352096    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp4pr\" (UniqueName: \"kubernetes.io/projected/8b94f66f-f022-414b-8784-603142645fde-kube-api-access-pp4pr\") pod \"maas-controller-58d867cfcf-zxv9g\" (UID: \"8b94f66f-f022-414b-8784-603142645fde\") " pod="opendatahub/maas-controller-58d867cfcf-zxv9g"
Jul 22 12:16:08.352326 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.352172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b94f66f-f022-414b-8784-603142645fde-webhook-cert\") pod \"maas-controller-58d867cfcf-zxv9g\" (UID: \"8b94f66f-f022-414b-8784-603142645fde\") " pod="opendatahub/maas-controller-58d867cfcf-zxv9g"
Jul 22 12:16:08.354780 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.354758    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b94f66f-f022-414b-8784-603142645fde-webhook-cert\") pod \"maas-controller-58d867cfcf-zxv9g\" (UID: \"8b94f66f-f022-414b-8784-603142645fde\") " pod="opendatahub/maas-controller-58d867cfcf-zxv9g"
Jul 22 12:16:08.363450 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.363431    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp4pr\" (UniqueName: \"kubernetes.io/projected/8b94f66f-f022-414b-8784-603142645fde-kube-api-access-pp4pr\") pod \"maas-controller-58d867cfcf-zxv9g\" (UID: \"8b94f66f-f022-414b-8784-603142645fde\") " pod="opendatahub/maas-controller-58d867cfcf-zxv9g"
Jul 22 12:16:08.476610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.476531    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58d867cfcf-zxv9g"
Jul 22 12:16:08.622099 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:08.622077    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58d867cfcf-zxv9g"]
Jul 22 12:16:08.624083 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:16:08.624056    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b94f66f_f022_414b_8784_603142645fde.slice/crio-f30fb0d7962ab28169be209a0eaf0c0db1929c6236d346f18ce3c0e1d815e364 WatchSource:0}: Error finding container f30fb0d7962ab28169be209a0eaf0c0db1929c6236d346f18ce3c0e1d815e364: Status 404 returned error can't find the container with id f30fb0d7962ab28169be209a0eaf0c0db1929c6236d346f18ce3c0e1d815e364
Jul 22 12:16:09.242508 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:09.242473    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d867cfcf-zxv9g" event={"ID":"8b94f66f-f022-414b-8784-603142645fde","Type":"ContainerStarted","Data":"308da6324294d86420d559dd2233c2754ccf0f303e28f51282b5cc253005c96c"}
Jul 22 12:16:09.242912 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:09.242568    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58d867cfcf-zxv9g" event={"ID":"8b94f66f-f022-414b-8784-603142645fde","Type":"ContainerStarted","Data":"f30fb0d7962ab28169be209a0eaf0c0db1929c6236d346f18ce3c0e1d815e364"}
Jul 22 12:16:09.242912 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:09.242593    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58d867cfcf-zxv9g"
Jul 22 12:16:20.250893 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:20.250860    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58d867cfcf-zxv9g"
Jul 22 12:16:20.271781 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:20.271731    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58d867cfcf-zxv9g" podStartSLOduration=11.877687018 podStartE2EDuration="12.271716154s" totalImagesPullingTime="394.029136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:16:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:16:09.269467444 +0000 UTC m=+858.291489160" watchObservedRunningTime="2026-07-22 12:16:20.271716154 +0000 UTC m=+869.293737855"
Jul 22 12:16:51.484184 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:16:51.480041    2584 scope.go:117] "RemoveContainer" containerID="fc5aade24728ff03919d818f3e9787f86574748aa31465c8c59f39bd7cc54965"
Jul 22 12:20:51.574887 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:51.574857    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"]
Jul 22 12:20:51.575441 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:51.575052    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb" podUID="eb9a229d-c58f-4b37-9083-aa80f5f4a81e" containerName="manager" containerID="cri-o://6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9" gracePeriod=10
Jul 22 12:20:54.527980 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.527951    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:20:54.564611 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.564575    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t77n\" (UniqueName: \"kubernetes.io/projected/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-kube-api-access-2t77n\") pod \"eb9a229d-c58f-4b37-9083-aa80f5f4a81e\" (UID: \"eb9a229d-c58f-4b37-9083-aa80f5f4a81e\") "
Jul 22 12:20:54.564781 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.564713    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-extensions-socket-volume\") pod \"eb9a229d-c58f-4b37-9083-aa80f5f4a81e\" (UID: \"eb9a229d-c58f-4b37-9083-aa80f5f4a81e\") "
Jul 22 12:20:54.565281 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.565252    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "eb9a229d-c58f-4b37-9083-aa80f5f4a81e" (UID: "eb9a229d-c58f-4b37-9083-aa80f5f4a81e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:20:54.566630 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.566601    2584 generic.go:356] "Generic (PLEG): container finished" podID="eb9a229d-c58f-4b37-9083-aa80f5f4a81e" containerID="6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9" exitCode=0
Jul 22 12:20:54.566732 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.566644    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"
Jul 22 12:20:54.566732 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.566669    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb" event={"ID":"eb9a229d-c58f-4b37-9083-aa80f5f4a81e","Type":"ContainerDied","Data":"6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9"}
Jul 22 12:20:54.566732 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.566724    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb" event={"ID":"eb9a229d-c58f-4b37-9083-aa80f5f4a81e","Type":"ContainerDied","Data":"6ba5dd19970444f45811a3a26b42c274efd6f29534331bfa8f98921a0ca15a8d"}
Jul 22 12:20:54.566895 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.566755    2584 scope.go:117] "RemoveContainer" containerID="6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9"
Jul 22 12:20:54.567681 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.567641    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-kube-api-access-2t77n" (OuterVolumeSpecName: "kube-api-access-2t77n") pod "eb9a229d-c58f-4b37-9083-aa80f5f4a81e" (UID: "eb9a229d-c58f-4b37-9083-aa80f5f4a81e"). InnerVolumeSpecName "kube-api-access-2t77n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:20:54.582805 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.582775    2584 scope.go:117] "RemoveContainer" containerID="6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9"
Jul 22 12:20:54.583202 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:20:54.583183    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9\": container with ID starting with 6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9 not found: ID does not exist" containerID="6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9"
Jul 22 12:20:54.583285 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.583217    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9"} err="failed to get container status \"6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9\": rpc error: code = NotFound desc = could not find container \"6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9\": container with ID starting with 6f13dcb662ea18832e6d8108df6a195e2746566899d72d7bdb8949f9cc2755e9 not found: ID does not exist"
Jul 22 12:20:54.666428 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.666372    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t77n\" (UniqueName: \"kubernetes.io/projected/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-kube-api-access-2t77n\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:20:54.666428 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.666421    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb9a229d-c58f-4b37-9083-aa80f5f4a81e-extensions-socket-volume\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:20:54.890319 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.890288    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"]
Jul 22 12:20:54.896792 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:54.896769    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c9cqb"]
Jul 22 12:20:55.525411 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:20:55.525365    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb9a229d-c58f-4b37-9083-aa80f5f4a81e" path="/var/lib/kubelet/pods/eb9a229d-c58f-4b37-9083-aa80f5f4a81e/volumes"
Jul 22 12:21:52.762439 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:52.762403    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"]
Jul 22 12:21:52.762870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:52.762785    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb9a229d-c58f-4b37-9083-aa80f5f4a81e" containerName="manager"
Jul 22 12:21:52.762870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:52.762801    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb9a229d-c58f-4b37-9083-aa80f5f4a81e" containerName="manager"
Jul 22 12:21:52.762870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:52.762855    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb9a229d-c58f-4b37-9083-aa80f5f4a81e" containerName="manager"
Jul 22 12:21:52.772759 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:52.772734    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"
Jul 22 12:21:52.775851 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:52.775828    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wtnkv\""
Jul 22 12:21:52.782621 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:52.782598    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"]
Jul 22 12:21:52.908894 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:52.908865    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c961749-41f2-4f28-b0a1-cc672cb297a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nvc27\" (UID: \"6c961749-41f2-4f28-b0a1-cc672cb297a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"
Jul 22 12:21:52.908894 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:52.908896    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h5sk\" (UniqueName: \"kubernetes.io/projected/6c961749-41f2-4f28-b0a1-cc672cb297a2-kube-api-access-2h5sk\") pod \"kuadrant-operator-controller-manager-86878c769f-nvc27\" (UID: \"6c961749-41f2-4f28-b0a1-cc672cb297a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"
Jul 22 12:21:53.009765 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:53.009727    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c961749-41f2-4f28-b0a1-cc672cb297a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nvc27\" (UID: \"6c961749-41f2-4f28-b0a1-cc672cb297a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"
Jul 22 12:21:53.009765 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:53.009765    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h5sk\" (UniqueName: \"kubernetes.io/projected/6c961749-41f2-4f28-b0a1-cc672cb297a2-kube-api-access-2h5sk\") pod \"kuadrant-operator-controller-manager-86878c769f-nvc27\" (UID: \"6c961749-41f2-4f28-b0a1-cc672cb297a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"
Jul 22 12:21:53.010194 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:53.010174    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c961749-41f2-4f28-b0a1-cc672cb297a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nvc27\" (UID: \"6c961749-41f2-4f28-b0a1-cc672cb297a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"
Jul 22 12:21:53.019423 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:53.019327    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h5sk\" (UniqueName: \"kubernetes.io/projected/6c961749-41f2-4f28-b0a1-cc672cb297a2-kube-api-access-2h5sk\") pod \"kuadrant-operator-controller-manager-86878c769f-nvc27\" (UID: \"6c961749-41f2-4f28-b0a1-cc672cb297a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"
Jul 22 12:21:53.084231 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:53.084200    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"
Jul 22 12:21:53.226617 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:53.226588    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"]
Jul 22 12:21:53.229914 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:21:53.229875    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c961749_41f2_4f28_b0a1_cc672cb297a2.slice/crio-250177572f6ba2a42e6f737aeee3f98979826e624572b1f238f089d8cb068603 WatchSource:0}: Error finding container 250177572f6ba2a42e6f737aeee3f98979826e624572b1f238f089d8cb068603: Status 404 returned error can't find the container with id 250177572f6ba2a42e6f737aeee3f98979826e624572b1f238f089d8cb068603
Jul 22 12:21:53.836621 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:53.836579    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27" event={"ID":"6c961749-41f2-4f28-b0a1-cc672cb297a2","Type":"ContainerStarted","Data":"2c04bba666c6da97d50da7ae8797413df3add7f8df79a189a907c7db7123be9e"}
Jul 22 12:21:53.837107 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:53.836642    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27" event={"ID":"6c961749-41f2-4f28-b0a1-cc672cb297a2","Type":"ContainerStarted","Data":"250177572f6ba2a42e6f737aeee3f98979826e624572b1f238f089d8cb068603"}
Jul 22 12:21:53.837107 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:53.836672    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"
Jul 22 12:21:53.861316 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:21:53.861267    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27" podStartSLOduration=1.861251 podStartE2EDuration="1.861251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:21:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:21:53.85977331 +0000 UTC m=+1202.881795012" watchObservedRunningTime="2026-07-22 12:21:53.861251 +0000 UTC m=+1202.883272709"
Jul 22 12:22:04.844037 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:22:04.844006    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvc27"
Jul 22 12:28:46.187326 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.187238    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"]
Jul 22 12:28:46.191219 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.191180    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:46.193976 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.193953    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-20d1eb60-serving-cert\""
Jul 22 12:28:46.194117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.194032    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-svscs\""
Jul 22 12:28:46.203408 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.203355    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"]
Jul 22 12:28:46.346512 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.346471    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ca03047-f892-437e-8fff-865a73c81e28-maas-api-tls\") pod \"maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:46.346695 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.346619    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9995c\" (UniqueName: \"kubernetes.io/projected/9ca03047-f892-437e-8fff-865a73c81e28-kube-api-access-9995c\") pod \"maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:46.346695 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.346677    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ca03047-f892-437e-8fff-865a73c81e28-openshift-service-ca\") pod \"maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:46.447818 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.447731    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9995c\" (UniqueName: \"kubernetes.io/projected/9ca03047-f892-437e-8fff-865a73c81e28-kube-api-access-9995c\") pod \"maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:46.447818 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.447786    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ca03047-f892-437e-8fff-865a73c81e28-openshift-service-ca\") pod \"maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:46.448019 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.447831    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ca03047-f892-437e-8fff-865a73c81e28-maas-api-tls\") pod \"maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:46.448593 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.448574    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ca03047-f892-437e-8fff-865a73c81e28-openshift-service-ca\") pod \"maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:46.450281 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.450251    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ca03047-f892-437e-8fff-865a73c81e28-maas-api-tls\") pod \"maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:46.463010 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.462985    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9995c\" (UniqueName: \"kubernetes.io/projected/9ca03047-f892-437e-8fff-865a73c81e28-kube-api-access-9995c\") pod \"maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:46.502632 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.502603    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:46.655106 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.655077    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"]
Jul 22 12:28:46.657294 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:28:46.657265    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca03047_f892_437e_8fff_865a73c81e28.slice/crio-664162e2a182c0d197b747b07b46a2c43433758ee3fa3dc0a781ad565f4f2d1e WatchSource:0}: Error finding container 664162e2a182c0d197b747b07b46a2c43433758ee3fa3dc0a781ad565f4f2d1e: Status 404 returned error can't find the container with id 664162e2a182c0d197b747b07b46a2c43433758ee3fa3dc0a781ad565f4f2d1e
Jul 22 12:28:46.658940 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.658922    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:28:46.766152 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:46.766064    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx" event={"ID":"9ca03047-f892-437e-8fff-865a73c81e28","Type":"ContainerStarted","Data":"664162e2a182c0d197b747b07b46a2c43433758ee3fa3dc0a781ad565f4f2d1e"}
Jul 22 12:28:47.772762 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:47.772676    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx" event={"ID":"9ca03047-f892-437e-8fff-865a73c81e28","Type":"ContainerStarted","Data":"52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f"}
Jul 22 12:28:47.773131 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:47.772785    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:28:47.793657 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:47.793611    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx" podStartSLOduration=0.973103905 podStartE2EDuration="1.793595117s" totalImagesPullingTime="820.491212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:28:47.792511779 +0000 UTC m=+1616.814533497" watchObservedRunningTime="2026-07-22 12:28:47.793595117 +0000 UTC m=+1616.815617023"
Jul 22 12:28:53.782681 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:28:53.782650    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:29:22.307340 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.307307    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"]
Jul 22 12:29:22.307801 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.307560    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx" podUID="9ca03047-f892-437e-8fff-865a73c81e28" containerName="maas-api" containerID="cri-o://52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f" gracePeriod=30
Jul 22 12:29:22.559216 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.559155    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:29:22.686281 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.686247    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ca03047-f892-437e-8fff-865a73c81e28-openshift-service-ca\") pod \"9ca03047-f892-437e-8fff-865a73c81e28\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") "
Jul 22 12:29:22.686474 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.686327    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ca03047-f892-437e-8fff-865a73c81e28-maas-api-tls\") pod \"9ca03047-f892-437e-8fff-865a73c81e28\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") "
Jul 22 12:29:22.686551 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.686474    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9995c\" (UniqueName: \"kubernetes.io/projected/9ca03047-f892-437e-8fff-865a73c81e28-kube-api-access-9995c\") pod \"9ca03047-f892-437e-8fff-865a73c81e28\" (UID: \"9ca03047-f892-437e-8fff-865a73c81e28\") "
Jul 22 12:29:22.686676 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.686655    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ca03047-f892-437e-8fff-865a73c81e28-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ca03047-f892-437e-8fff-865a73c81e28" (UID: "9ca03047-f892-437e-8fff-865a73c81e28"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:29:22.686769 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.686753    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ca03047-f892-437e-8fff-865a73c81e28-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:29:22.688451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.688424    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca03047-f892-437e-8fff-865a73c81e28-kube-api-access-9995c" (OuterVolumeSpecName: "kube-api-access-9995c") pod "9ca03047-f892-437e-8fff-865a73c81e28" (UID: "9ca03047-f892-437e-8fff-865a73c81e28"). InnerVolumeSpecName "kube-api-access-9995c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:29:22.688571 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.688457    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ca03047-f892-437e-8fff-865a73c81e28-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ca03047-f892-437e-8fff-865a73c81e28" (UID: "9ca03047-f892-437e-8fff-865a73c81e28"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:29:22.787814 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.787768    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9995c\" (UniqueName: \"kubernetes.io/projected/9ca03047-f892-437e-8fff-865a73c81e28-kube-api-access-9995c\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:29:22.787814 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.787808    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ca03047-f892-437e-8fff-865a73c81e28-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:29:22.939534 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.939505    2584 generic.go:356] "Generic (PLEG): container finished" podID="9ca03047-f892-437e-8fff-865a73c81e28" containerID="52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f" exitCode=0
Jul 22 12:29:22.939742 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.939565    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"
Jul 22 12:29:22.939742 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.939590    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx" event={"ID":"9ca03047-f892-437e-8fff-865a73c81e28","Type":"ContainerDied","Data":"52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f"}
Jul 22 12:29:22.939742 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.939655    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx" event={"ID":"9ca03047-f892-437e-8fff-865a73c81e28","Type":"ContainerDied","Data":"664162e2a182c0d197b747b07b46a2c43433758ee3fa3dc0a781ad565f4f2d1e"}
Jul 22 12:29:22.939742 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.939698    2584 scope.go:117] "RemoveContainer" containerID="52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f"
Jul 22 12:29:22.948429 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.948399    2584 scope.go:117] "RemoveContainer" containerID="52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f"
Jul 22 12:29:22.948711 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:29:22.948688    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f\": container with ID starting with 52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f not found: ID does not exist" containerID="52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f"
Jul 22 12:29:22.948799 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.948721    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f"} err="failed to get container status \"52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f\": rpc error: code = NotFound desc = could not find container \"52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f\": container with ID starting with 52b64e64885293e28226db59872c6b38925e14fcc3650566b121e69ea5836d9f not found: ID does not exist"
Jul 22 12:29:22.966412 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.966364    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"]
Jul 22 12:29:22.970694 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:22.970673    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-20d1eb60-5bcd58fc8f-rbzvx"]
Jul 22 12:29:23.525593 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:23.525560    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ca03047-f892-437e-8fff-865a73c81e28" path="/var/lib/kubelet/pods/9ca03047-f892-437e-8fff-865a73c81e28/volumes"
Jul 22 12:29:28.251138 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.251104    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"]
Jul 22 12:29:28.251746 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.251689    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ca03047-f892-437e-8fff-865a73c81e28" containerName="maas-api"
Jul 22 12:29:28.251746 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.251708    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ca03047-f892-437e-8fff-865a73c81e28" containerName="maas-api"
Jul 22 12:29:28.251872 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.251808    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ca03047-f892-437e-8fff-865a73c81e28" containerName="maas-api"
Jul 22 12:29:28.256176 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.256156    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.258881 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.258860    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-0efd59a1-gw-openshift-default-dockercfg-4vpmt\""
Jul 22 12:29:28.260912 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.260887    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"]
Jul 22 12:29:28.340114 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.340071    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-token\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.340291 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.340124    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istiod-ca-cert\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.340291 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.340160    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2rmd\" (UniqueName: \"kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-kube-api-access-m2rmd\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.340291 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.340225    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-credential-socket\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.340291 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.340255    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-socket\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.340291 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.340276    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-envoy\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.340704 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.340295    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-podinfo\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.340704 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.340319    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-certs\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.340704 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.340356    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-data\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.441761 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.441725    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-envoy\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.441761 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.441764    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-podinfo\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.442026 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.441795    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-certs\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.442026 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.441830    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-data\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.442026 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.441915    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-token\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.442026 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.441948    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istiod-ca-cert\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.442026 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.441976    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rmd\" (UniqueName: \"kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-kube-api-access-m2rmd\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.442343 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.442121    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-credential-socket\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.442343 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.442164    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-socket\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.442343 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.442251    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-certs\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.442600 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.442578    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-credential-socket\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.442827 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.442809    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-socket\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.442964 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.442943    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istiod-ca-cert\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.443000 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.442988    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-data\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.444389 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.444355    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-envoy\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.444495 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.444446    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-podinfo\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.453805 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.453776    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-token\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.462356 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.462336    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2rmd\" (UniqueName: \"kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-kube-api-access-m2rmd\") pod \"e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") " pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.569571 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.569482    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:28.708002 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.707974    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"]
Jul 22 12:29:28.709854 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:29:28.709824    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b008d6_d377_4b75_9c77_50855ac1f3a6.slice/crio-fb84dd5a48acff4d811a6d8ae97c4bc2ca705811e58036e97d648e718b965997 WatchSource:0}: Error finding container fb84dd5a48acff4d811a6d8ae97c4bc2ca705811e58036e97d648e718b965997: Status 404 returned error can't find the container with id fb84dd5a48acff4d811a6d8ae97c4bc2ca705811e58036e97d648e718b965997
Jul 22 12:29:28.711850 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.711815    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:29:28.711924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.711878    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:29:28.711924 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.711908    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:29:28.973445 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.973411    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" event={"ID":"c4b008d6-d377-4b75-9c77-50855ac1f3a6","Type":"ContainerStarted","Data":"92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e"}
Jul 22 12:29:28.973618 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.973466    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" event={"ID":"c4b008d6-d377-4b75-9c77-50855ac1f3a6","Type":"ContainerStarted","Data":"fb84dd5a48acff4d811a6d8ae97c4bc2ca705811e58036e97d648e718b965997"}
Jul 22 12:29:28.995329 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:28.995282    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" podStartSLOduration=0.995267197 podStartE2EDuration="995.267197ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:29:28.993129326 +0000 UTC m=+1658.015151045" watchObservedRunningTime="2026-07-22 12:29:28.995267197 +0000 UTC m=+1658.017288898"
Jul 22 12:29:29.569998 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:29.569967    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:29.575328 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:29.575304    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:29.978334 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:29.978303    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:29.979221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:29.979203    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:30.558083 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.558048    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"]
Jul 22 12:29:30.562449 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.562424    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:30.564932 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.564911    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-0efd59a1-serving-cert\""
Jul 22 12:29:30.565111 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.565015    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-svscs\""
Jul 22 12:29:30.570565 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.570487    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"]
Jul 22 12:29:30.661904 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.661867    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20883670-43c6-46dc-93ac-bf37c26934aa-maas-api-tls\") pod \"maas-api-e2e-migrate-0efd59a1-659875c789-mp78c\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:30.662114 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.662083    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z6tv\" (UniqueName: \"kubernetes.io/projected/20883670-43c6-46dc-93ac-bf37c26934aa-kube-api-access-8z6tv\") pod \"maas-api-e2e-migrate-0efd59a1-659875c789-mp78c\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:30.662198 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.662129    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20883670-43c6-46dc-93ac-bf37c26934aa-openshift-service-ca\") pod \"maas-api-e2e-migrate-0efd59a1-659875c789-mp78c\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:30.763287 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.763250    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20883670-43c6-46dc-93ac-bf37c26934aa-maas-api-tls\") pod \"maas-api-e2e-migrate-0efd59a1-659875c789-mp78c\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:30.763528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.763345    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z6tv\" (UniqueName: \"kubernetes.io/projected/20883670-43c6-46dc-93ac-bf37c26934aa-kube-api-access-8z6tv\") pod \"maas-api-e2e-migrate-0efd59a1-659875c789-mp78c\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:30.763528 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.763371    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20883670-43c6-46dc-93ac-bf37c26934aa-openshift-service-ca\") pod \"maas-api-e2e-migrate-0efd59a1-659875c789-mp78c\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:30.763528 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:29:30.763432    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-serving-cert: secret "maas-api-e2e-migrate-0efd59a1-serving-cert" not found
Jul 22 12:29:30.763528 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:29:30.763504    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20883670-43c6-46dc-93ac-bf37c26934aa-maas-api-tls podName:20883670-43c6-46dc-93ac-bf37c26934aa nodeName:}" failed. No retries permitted until 2026-07-22 12:29:31.263481843 +0000 UTC m=+1660.285503538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/20883670-43c6-46dc-93ac-bf37c26934aa-maas-api-tls") pod "maas-api-e2e-migrate-0efd59a1-659875c789-mp78c" (UID: "20883670-43c6-46dc-93ac-bf37c26934aa") : secret "maas-api-e2e-migrate-0efd59a1-serving-cert" not found
Jul 22 12:29:30.763979 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.763963    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20883670-43c6-46dc-93ac-bf37c26934aa-openshift-service-ca\") pod \"maas-api-e2e-migrate-0efd59a1-659875c789-mp78c\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:30.773305 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:30.773278    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z6tv\" (UniqueName: \"kubernetes.io/projected/20883670-43c6-46dc-93ac-bf37c26934aa-kube-api-access-8z6tv\") pod \"maas-api-e2e-migrate-0efd59a1-659875c789-mp78c\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:31.269489 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:31.269452    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20883670-43c6-46dc-93ac-bf37c26934aa-maas-api-tls\") pod \"maas-api-e2e-migrate-0efd59a1-659875c789-mp78c\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:31.271973 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:31.271943    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20883670-43c6-46dc-93ac-bf37c26934aa-maas-api-tls\") pod \"maas-api-e2e-migrate-0efd59a1-659875c789-mp78c\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:31.480402 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:31.480348    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:31.623239 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:31.623215    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"]
Jul 22 12:29:31.624507 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:29:31.624481    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20883670_43c6_46dc_93ac_bf37c26934aa.slice/crio-1f1202718cf550e0dba43d66fe56bb8eded431403475ad7821d102f81b77aeef WatchSource:0}: Error finding container 1f1202718cf550e0dba43d66fe56bb8eded431403475ad7821d102f81b77aeef: Status 404 returned error can't find the container with id 1f1202718cf550e0dba43d66fe56bb8eded431403475ad7821d102f81b77aeef
Jul 22 12:29:31.995733 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:31.995692    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c" event={"ID":"20883670-43c6-46dc-93ac-bf37c26934aa","Type":"ContainerStarted","Data":"1f1202718cf550e0dba43d66fe56bb8eded431403475ad7821d102f81b77aeef"}
Jul 22 12:29:33.002694 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:33.002655    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c" event={"ID":"20883670-43c6-46dc-93ac-bf37c26934aa","Type":"ContainerStarted","Data":"908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e"}
Jul 22 12:29:33.003150 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:33.002754    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:33.022998 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:33.022934    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c" podStartSLOduration=2.707783327 podStartE2EDuration="3.02291847s" totalImagesPullingTime="315.135143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:29:33.020946028 +0000 UTC m=+1662.042967730" watchObservedRunningTime="2026-07-22 12:29:33.02291847 +0000 UTC m=+1662.044940211"
Jul 22 12:29:39.014114 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:39.014083    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:29:44.980527 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:44.980488    2584 patch_prober.go:28] interesting pod/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 22 12:29:44.980964 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:44.980561    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 22 12:29:45.065145 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:45.065111    2584 generic.go:356] "Generic (PLEG): container finished" podID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerID="92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e" exitCode=0
Jul 22 12:29:45.065306 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:45.065184    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" event={"ID":"c4b008d6-d377-4b75-9c77-50855ac1f3a6","Type":"ContainerDied","Data":"92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e"}
Jul 22 12:29:45.065684 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:45.065670    2584 scope.go:117] "RemoveContainer" containerID="92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e"
Jul 22 12:29:45.066762 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:45.066732    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:29:45.066833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:45.066818    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:29:45.066879 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:45.066866    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:29:46.071711 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:46.071681    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" event={"ID":"c4b008d6-d377-4b75-9c77-50855ac1f3a6","Type":"ContainerStarted","Data":"7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e"}
Jul 22 12:29:46.570555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:46.570521    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:46.570737 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:46.570716    2584 patch_prober.go:28] interesting pod/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 22 12:29:46.570784 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:46.570765    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 22 12:29:47.570471 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:47.570431    2584 patch_prober.go:28] interesting pod/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 22 12:29:47.570974 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:47.570487    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 22 12:29:48.570739 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:48.570705    2584 patch_prober.go:28] interesting pod/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 22 12:29:48.571144 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:48.570760    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 22 12:29:49.570437 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:49.570401    2584 patch_prober.go:28] interesting pod/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 22 12:29:49.570624 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:49.570460    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 22 12:29:50.573587 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:50.573558    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:50.574054 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:50.573948    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:29:50.574608 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:29:50.574588    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:30:06.204589 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.204554    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"]
Jul 22 12:30:06.205305 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.204764    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c" podUID="20883670-43c6-46dc-93ac-bf37c26934aa" containerName="maas-api" containerID="cri-o://908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e" gracePeriod=30
Jul 22 12:30:06.460573 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.460504    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:30:06.613817 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.613786    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z6tv\" (UniqueName: \"kubernetes.io/projected/20883670-43c6-46dc-93ac-bf37c26934aa-kube-api-access-8z6tv\") pod \"20883670-43c6-46dc-93ac-bf37c26934aa\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") "
Jul 22 12:30:06.613989 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.613841    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20883670-43c6-46dc-93ac-bf37c26934aa-openshift-service-ca\") pod \"20883670-43c6-46dc-93ac-bf37c26934aa\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") "
Jul 22 12:30:06.613989 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.613887    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20883670-43c6-46dc-93ac-bf37c26934aa-maas-api-tls\") pod \"20883670-43c6-46dc-93ac-bf37c26934aa\" (UID: \"20883670-43c6-46dc-93ac-bf37c26934aa\") "
Jul 22 12:30:06.614254 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.614223    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20883670-43c6-46dc-93ac-bf37c26934aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20883670-43c6-46dc-93ac-bf37c26934aa" (UID: "20883670-43c6-46dc-93ac-bf37c26934aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:30:06.615967 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.615945    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20883670-43c6-46dc-93ac-bf37c26934aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20883670-43c6-46dc-93ac-bf37c26934aa" (UID: "20883670-43c6-46dc-93ac-bf37c26934aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:30:06.616055 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.616033    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20883670-43c6-46dc-93ac-bf37c26934aa-kube-api-access-8z6tv" (OuterVolumeSpecName: "kube-api-access-8z6tv") pod "20883670-43c6-46dc-93ac-bf37c26934aa" (UID: "20883670-43c6-46dc-93ac-bf37c26934aa"). InnerVolumeSpecName "kube-api-access-8z6tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:30:06.715256 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.715173    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z6tv\" (UniqueName: \"kubernetes.io/projected/20883670-43c6-46dc-93ac-bf37c26934aa-kube-api-access-8z6tv\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:06.715256 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.715198    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20883670-43c6-46dc-93ac-bf37c26934aa-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:06.715256 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:06.715211    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20883670-43c6-46dc-93ac-bf37c26934aa-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:07.179027 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:07.178996    2584 generic.go:356] "Generic (PLEG): container finished" podID="20883670-43c6-46dc-93ac-bf37c26934aa" containerID="908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e" exitCode=0
Jul 22 12:30:07.179027 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:07.179033    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c" event={"ID":"20883670-43c6-46dc-93ac-bf37c26934aa","Type":"ContainerDied","Data":"908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e"}
Jul 22 12:30:07.179237 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:07.179052    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"
Jul 22 12:30:07.179237 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:07.179097    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c" event={"ID":"20883670-43c6-46dc-93ac-bf37c26934aa","Type":"ContainerDied","Data":"1f1202718cf550e0dba43d66fe56bb8eded431403475ad7821d102f81b77aeef"}
Jul 22 12:30:07.179237 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:07.179119    2584 scope.go:117] "RemoveContainer" containerID="908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e"
Jul 22 12:30:07.188922 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:07.188904    2584 scope.go:117] "RemoveContainer" containerID="908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e"
Jul 22 12:30:07.189164 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:30:07.189143    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e\": container with ID starting with 908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e not found: ID does not exist" containerID="908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e"
Jul 22 12:30:07.189238 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:07.189171    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e"} err="failed to get container status \"908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e\": rpc error: code = NotFound desc = could not find container \"908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e\": container with ID starting with 908df52b777617db6d56495da383e157f0ab84b86e8700d87004ac803b20c25e not found: ID does not exist"
Jul 22 12:30:07.203750 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:07.203725    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"]
Jul 22 12:30:07.208713 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:07.208689    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0efd59a1-659875c789-mp78c"]
Jul 22 12:30:07.525613 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:07.525537    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20883670-43c6-46dc-93ac-bf37c26934aa" path="/var/lib/kubelet/pods/20883670-43c6-46dc-93ac-bf37c26934aa/volumes"
Jul 22 12:30:10.808416 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:10.808371    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"]
Jul 22 12:30:10.808814 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:10.808593    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy" containerID="cri-o://7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e" gracePeriod=30
Jul 22 12:30:14.630056 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.630022    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"]
Jul 22 12:30:14.630560 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.630436    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20883670-43c6-46dc-93ac-bf37c26934aa" containerName="maas-api"
Jul 22 12:30:14.630560 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.630447    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="20883670-43c6-46dc-93ac-bf37c26934aa" containerName="maas-api"
Jul 22 12:30:14.630560 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.630519    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="20883670-43c6-46dc-93ac-bf37c26934aa" containerName="maas-api"
Jul 22 12:30:14.633410 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.633391    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:14.636254 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.636232    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-cf37a858-serving-cert\""
Jul 22 12:30:14.636254 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.636241    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-svscs\""
Jul 22 12:30:14.644038 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.644017    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"]
Jul 22 12:30:14.792658 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.792621    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-openshift-service-ca\") pod \"maas-api-e2e-ait-cf37a858-6cc547db7d-582t7\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:14.792828 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.792665    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9jr7\" (UniqueName: \"kubernetes.io/projected/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-kube-api-access-q9jr7\") pod \"maas-api-e2e-ait-cf37a858-6cc547db7d-582t7\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:14.792828 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.792701    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-maas-api-tls\") pod \"maas-api-e2e-ait-cf37a858-6cc547db7d-582t7\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:14.893429 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.893333    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-openshift-service-ca\") pod \"maas-api-e2e-ait-cf37a858-6cc547db7d-582t7\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:14.893429 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.893372    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9jr7\" (UniqueName: \"kubernetes.io/projected/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-kube-api-access-q9jr7\") pod \"maas-api-e2e-ait-cf37a858-6cc547db7d-582t7\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:14.893612 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.893440    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-maas-api-tls\") pod \"maas-api-e2e-ait-cf37a858-6cc547db7d-582t7\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:14.893612 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:30:14.893604    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-serving-cert: secret "maas-api-e2e-ait-cf37a858-serving-cert" not found
Jul 22 12:30:14.893679 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:30:14.893668    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-maas-api-tls podName:4c48cc4f-d0b1-4015-b33d-2dfe81e3f327 nodeName:}" failed. No retries permitted until 2026-07-22 12:30:15.393649652 +0000 UTC m=+1704.415671337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-maas-api-tls") pod "maas-api-e2e-ait-cf37a858-6cc547db7d-582t7" (UID: "4c48cc4f-d0b1-4015-b33d-2dfe81e3f327") : secret "maas-api-e2e-ait-cf37a858-serving-cert" not found
Jul 22 12:30:14.894026 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.893997    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-openshift-service-ca\") pod \"maas-api-e2e-ait-cf37a858-6cc547db7d-582t7\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:14.903403 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:14.903365    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9jr7\" (UniqueName: \"kubernetes.io/projected/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-kube-api-access-q9jr7\") pod \"maas-api-e2e-ait-cf37a858-6cc547db7d-582t7\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:15.398153 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:15.398122    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-maas-api-tls\") pod \"maas-api-e2e-ait-cf37a858-6cc547db7d-582t7\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:15.400548 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:15.400518    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-maas-api-tls\") pod \"maas-api-e2e-ait-cf37a858-6cc547db7d-582t7\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:15.546677 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:15.546645    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:15.693759 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:15.693726    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"]
Jul 22 12:30:15.695281 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:30:15.695258    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c48cc4f_d0b1_4015_b33d_2dfe81e3f327.slice/crio-3c0ccc96e6119e041cb7008ec657d073c3b1ee886bdbf46f482640d9c9372733 WatchSource:0}: Error finding container 3c0ccc96e6119e041cb7008ec657d073c3b1ee886bdbf46f482640d9c9372733: Status 404 returned error can't find the container with id 3c0ccc96e6119e041cb7008ec657d073c3b1ee886bdbf46f482640d9c9372733
Jul 22 12:30:16.066078 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.066057    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:30:16.204539 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.204510    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-podinfo\") pod \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") "
Jul 22 12:30:16.204680 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.204549    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2rmd\" (UniqueName: \"kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-kube-api-access-m2rmd\") pod \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") "
Jul 22 12:30:16.204736 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.204714    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-envoy\") pod \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") "
Jul 22 12:30:16.204787 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.204766    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-data\") pod \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") "
Jul 22 12:30:16.204842 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.204799    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-certs\") pod \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") "
Jul 22 12:30:16.204893 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.204854    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-token\") pod \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") "
Jul 22 12:30:16.204893 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.204888    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istiod-ca-cert\") pod \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") "
Jul 22 12:30:16.204996 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.204911    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-socket\") pod \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") "
Jul 22 12:30:16.204996 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.204942    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-credential-socket\") pod \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\" (UID: \"c4b008d6-d377-4b75-9c77-50855ac1f3a6\") "
Jul 22 12:30:16.205119 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.205069    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c4b008d6-d377-4b75-9c77-50855ac1f3a6" (UID: "c4b008d6-d377-4b75-9c77-50855ac1f3a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:30:16.205324 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.205267    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c4b008d6-d377-4b75-9c77-50855ac1f3a6" (UID: "c4b008d6-d377-4b75-9c77-50855ac1f3a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:30:16.205444 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.205322    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-certs\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:16.205680 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.205652    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c4b008d6-d377-4b75-9c77-50855ac1f3a6" (UID: "c4b008d6-d377-4b75-9c77-50855ac1f3a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:30:16.205806 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.205699    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c4b008d6-d377-4b75-9c77-50855ac1f3a6" (UID: "c4b008d6-d377-4b75-9c77-50855ac1f3a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:30:16.206171 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.206150    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "c4b008d6-d377-4b75-9c77-50855ac1f3a6" (UID: "c4b008d6-d377-4b75-9c77-50855ac1f3a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:30:16.206876 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.206859    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-kube-api-access-m2rmd" (OuterVolumeSpecName: "kube-api-access-m2rmd") pod "c4b008d6-d377-4b75-9c77-50855ac1f3a6" (UID: "c4b008d6-d377-4b75-9c77-50855ac1f3a6"). InnerVolumeSpecName "kube-api-access-m2rmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:30:16.206989 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.206975    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c4b008d6-d377-4b75-9c77-50855ac1f3a6" (UID: "c4b008d6-d377-4b75-9c77-50855ac1f3a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:30:16.207186 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.207174    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c4b008d6-d377-4b75-9c77-50855ac1f3a6" (UID: "c4b008d6-d377-4b75-9c77-50855ac1f3a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:30:16.207422 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.207398    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "c4b008d6-d377-4b75-9c77-50855ac1f3a6" (UID: "c4b008d6-d377-4b75-9c77-50855ac1f3a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:30:16.224907 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.224879    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7" event={"ID":"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327","Type":"ContainerStarted","Data":"5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5"}
Jul 22 12:30:16.225068 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.225051    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:16.225135 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.225094    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7" event={"ID":"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327","Type":"ContainerStarted","Data":"3c0ccc96e6119e041cb7008ec657d073c3b1ee886bdbf46f482640d9c9372733"}
Jul 22 12:30:16.226426 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.226407    2584 generic.go:356] "Generic (PLEG): container finished" podID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerID="7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e" exitCode=0
Jul 22 12:30:16.226504 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.226439    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" event={"ID":"c4b008d6-d377-4b75-9c77-50855ac1f3a6","Type":"ContainerDied","Data":"7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e"}
Jul 22 12:30:16.226504 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.226445    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"
Jul 22 12:30:16.226504 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.226467    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq" event={"ID":"c4b008d6-d377-4b75-9c77-50855ac1f3a6","Type":"ContainerDied","Data":"fb84dd5a48acff4d811a6d8ae97c4bc2ca705811e58036e97d648e718b965997"}
Jul 22 12:30:16.226504 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.226489    2584 scope.go:117] "RemoveContainer" containerID="7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e"
Jul 22 12:30:16.235479 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.235462    2584 scope.go:117] "RemoveContainer" containerID="92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e"
Jul 22 12:30:16.243275 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.243260    2584 scope.go:117] "RemoveContainer" containerID="7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e"
Jul 22 12:30:16.244433 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:30:16.243943    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e\": container with ID starting with 7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e not found: ID does not exist" containerID="7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e"
Jul 22 12:30:16.244433 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.243978    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e"} err="failed to get container status \"7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e\": rpc error: code = NotFound desc = could not find container \"7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e\": container with ID starting with 7775e7e6897cbc1a3f9dff60669b6aaafceb199c9a2c558215b29d08ffb5167e not found: ID does not exist"
Jul 22 12:30:16.244433 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.244002    2584 scope.go:117] "RemoveContainer" containerID="92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e"
Jul 22 12:30:16.244433 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:30:16.244240    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e\": container with ID starting with 92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e not found: ID does not exist" containerID="92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e"
Jul 22 12:30:16.244433 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.244271    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e"} err="failed to get container status \"92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e\": rpc error: code = NotFound desc = could not find container \"92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e\": container with ID starting with 92bf91ee5fa637124ddee74237cfcad919a9f34d33971224d853e6b135aed27e not found: ID does not exist"
Jul 22 12:30:16.245555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.245504    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7" podStartSLOduration=1.851032949 podStartE2EDuration="2.245486604s" totalImagesPullingTime="394.453655ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:30:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:30:16.243656204 +0000 UTC m=+1705.265677931" watchObservedRunningTime="2026-07-22 12:30:16.245486604 +0000 UTC m=+1705.267508305"
Jul 22 12:30:16.259581 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.259559    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"]
Jul 22 12:30:16.265648 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.265629    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-0efd59a1-gw-openshift-default-585895d895-69dpq"]
Jul 22 12:30:16.306303 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.306282    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-data\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:16.306303 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.306303    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-token\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:16.306449 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.306312    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istiod-ca-cert\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:16.306449 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.306322    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-workload-socket\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:16.306449 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.306330    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-credential-socket\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:16.306449 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.306340    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-podinfo\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:16.306449 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.306348    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2rmd\" (UniqueName: \"kubernetes.io/projected/c4b008d6-d377-4b75-9c77-50855ac1f3a6-kube-api-access-m2rmd\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:16.306449 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:16.306356    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b008d6-d377-4b75-9c77-50855ac1f3a6-istio-envoy\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:17.526092 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:17.526058    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" path="/var/lib/kubelet/pods/c4b008d6-d377-4b75-9c77-50855ac1f3a6/volumes"
Jul 22 12:30:22.240108 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:22.240080    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:50.967373 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:50.967340    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"]
Jul 22 12:30:50.967912 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:50.967568    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7" podUID="4c48cc4f-d0b1-4015-b33d-2dfe81e3f327" containerName="maas-api" containerID="cri-o://5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5" gracePeriod=30
Jul 22 12:30:51.217150 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.217129    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:51.334223 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.334143    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9jr7\" (UniqueName: \"kubernetes.io/projected/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-kube-api-access-q9jr7\") pod \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") "
Jul 22 12:30:51.334370 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.334267    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-openshift-service-ca\") pod \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") "
Jul 22 12:30:51.334370 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.334318    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-maas-api-tls\") pod \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\" (UID: \"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327\") "
Jul 22 12:30:51.334651 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.334630    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c48cc4f-d0b1-4015-b33d-2dfe81e3f327" (UID: "4c48cc4f-d0b1-4015-b33d-2dfe81e3f327"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:30:51.336321 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.336300    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c48cc4f-d0b1-4015-b33d-2dfe81e3f327" (UID: "4c48cc4f-d0b1-4015-b33d-2dfe81e3f327"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:30:51.336321 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.336296    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-kube-api-access-q9jr7" (OuterVolumeSpecName: "kube-api-access-q9jr7") pod "4c48cc4f-d0b1-4015-b33d-2dfe81e3f327" (UID: "4c48cc4f-d0b1-4015-b33d-2dfe81e3f327"). InnerVolumeSpecName "kube-api-access-q9jr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:30:51.395419 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.395391    2584 generic.go:356] "Generic (PLEG): container finished" podID="4c48cc4f-d0b1-4015-b33d-2dfe81e3f327" containerID="5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5" exitCode=0
Jul 22 12:30:51.395580 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.395440    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"
Jul 22 12:30:51.395580 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.395479    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7" event={"ID":"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327","Type":"ContainerDied","Data":"5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5"}
Jul 22 12:30:51.395580 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.395536    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7" event={"ID":"4c48cc4f-d0b1-4015-b33d-2dfe81e3f327","Type":"ContainerDied","Data":"3c0ccc96e6119e041cb7008ec657d073c3b1ee886bdbf46f482640d9c9372733"}
Jul 22 12:30:51.395580 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.395564    2584 scope.go:117] "RemoveContainer" containerID="5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5"
Jul 22 12:30:51.405060 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.405042    2584 scope.go:117] "RemoveContainer" containerID="5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5"
Jul 22 12:30:51.405282 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:30:51.405262    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5\": container with ID starting with 5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5 not found: ID does not exist" containerID="5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5"
Jul 22 12:30:51.405328 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.405290    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5"} err="failed to get container status \"5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5\": rpc error: code = NotFound desc = could not find container \"5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5\": container with ID starting with 5eaf26bfd20775c65caf29f430ccaad3d834122522fb40ea037987309ffbd4b5 not found: ID does not exist"
Jul 22 12:30:51.418259 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.418233    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"]
Jul 22 12:30:51.422189 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.422169    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cf37a858-6cc547db7d-582t7"]
Jul 22 12:30:51.435940 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.435922    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9jr7\" (UniqueName: \"kubernetes.io/projected/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-kube-api-access-q9jr7\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:51.436001 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.435940    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:51.436001 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.435950    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:30:51.525310 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:51.525282    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c48cc4f-d0b1-4015-b33d-2dfe81e3f327" path="/var/lib/kubelet/pods/4c48cc4f-d0b1-4015-b33d-2dfe81e3f327/volumes"
Jul 22 12:30:59.850705 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.850671    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"]
Jul 22 12:30:59.851151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.851033    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c48cc4f-d0b1-4015-b33d-2dfe81e3f327" containerName="maas-api"
Jul 22 12:30:59.851151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.851044    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c48cc4f-d0b1-4015-b33d-2dfe81e3f327" containerName="maas-api"
Jul 22 12:30:59.851151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.851054    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy"
Jul 22 12:30:59.851151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.851060    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy"
Jul 22 12:30:59.851151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.851074    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy"
Jul 22 12:30:59.851151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.851080    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy"
Jul 22 12:30:59.851151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.851141    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy"
Jul 22 12:30:59.851151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.851152    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c48cc4f-d0b1-4015-b33d-2dfe81e3f327" containerName="maas-api"
Jul 22 12:30:59.854147 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.854131    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:30:59.856905 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.856882    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-svscs\""
Jul 22 12:30:59.857049 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.856882    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a7ea4ba9-serving-cert\""
Jul 22 12:30:59.865481 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.865460    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"]
Jul 22 12:30:59.921755 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.921719    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-openshift-service-ca\") pod \"maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:30:59.921953 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.921869    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-maas-api-tls\") pod \"maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:30:59.921953 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:30:59.921912    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb6s7\" (UniqueName: \"kubernetes.io/projected/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-kube-api-access-bb6s7\") pod \"maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:00.023232 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:00.023193    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-maas-api-tls\") pod \"maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:00.023232 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:00.023230    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb6s7\" (UniqueName: \"kubernetes.io/projected/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-kube-api-access-bb6s7\") pod \"maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:00.023516 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:00.023265    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-openshift-service-ca\") pod \"maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:00.023516 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:31:00.023461    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-serving-cert: secret "maas-api-e2e-derive-a7ea4ba9-serving-cert" not found
Jul 22 12:31:00.023630 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:31:00.023532    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-maas-api-tls podName:1c2c82b6-06c2-46a6-ae3d-cf3b40d43072 nodeName:}" failed. No retries permitted until 2026-07-22 12:31:00.523516836 +0000 UTC m=+1749.545538516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-maas-api-tls") pod "maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd" (UID: "1c2c82b6-06c2-46a6-ae3d-cf3b40d43072") : secret "maas-api-e2e-derive-a7ea4ba9-serving-cert" not found
Jul 22 12:31:00.023912 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:00.023896    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-openshift-service-ca\") pod \"maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:00.034874 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:00.034842    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb6s7\" (UniqueName: \"kubernetes.io/projected/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-kube-api-access-bb6s7\") pod \"maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:00.528056 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:00.528023    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-maas-api-tls\") pod \"maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:00.530503 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:00.530485    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-maas-api-tls\") pod \"maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:00.765208 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:00.765174    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:00.908672 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:00.908598    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"]
Jul 22 12:31:00.912575 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:31:00.912544    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c2c82b6_06c2_46a6_ae3d_cf3b40d43072.slice/crio-d428218c14d13784d0ffa471b5a0eff58421bc89231f60f7bfebdae4fe09611c WatchSource:0}: Error finding container d428218c14d13784d0ffa471b5a0eff58421bc89231f60f7bfebdae4fe09611c: Status 404 returned error can't find the container with id d428218c14d13784d0ffa471b5a0eff58421bc89231f60f7bfebdae4fe09611c
Jul 22 12:31:01.442100 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:01.442067    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd" event={"ID":"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072","Type":"ContainerStarted","Data":"e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762"}
Jul 22 12:31:01.442212 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:01.442129    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd" event={"ID":"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072","Type":"ContainerStarted","Data":"d428218c14d13784d0ffa471b5a0eff58421bc89231f60f7bfebdae4fe09611c"}
Jul 22 12:31:01.442212 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:01.442168    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:01.462163 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:01.462114    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd" podStartSLOduration=2.001219363 podStartE2EDuration="2.462099215s" totalImagesPullingTime="460.879852ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:30:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:31:01.461043633 +0000 UTC m=+1750.483065336" watchObservedRunningTime="2026-07-22 12:31:01.462099215 +0000 UTC m=+1750.484120917"
Jul 22 12:31:07.453434 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:07.453371    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:35.509530 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.509492    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"]
Jul 22 12:31:35.510115 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.509838    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd" podUID="1c2c82b6-06c2-46a6-ae3d-cf3b40d43072" containerName="maas-api" containerID="cri-o://e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762" gracePeriod=30
Jul 22 12:31:35.757252 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.757226    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:35.857961 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.857933    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb6s7\" (UniqueName: \"kubernetes.io/projected/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-kube-api-access-bb6s7\") pod \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") "
Jul 22 12:31:35.858167 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.858065    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-maas-api-tls\") pod \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") "
Jul 22 12:31:35.858167 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.858098    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-openshift-service-ca\") pod \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\" (UID: \"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072\") "
Jul 22 12:31:35.858500 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.858473    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c2c82b6-06c2-46a6-ae3d-cf3b40d43072" (UID: "1c2c82b6-06c2-46a6-ae3d-cf3b40d43072"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:31:35.860002 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.859974    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-kube-api-access-bb6s7" (OuterVolumeSpecName: "kube-api-access-bb6s7") pod "1c2c82b6-06c2-46a6-ae3d-cf3b40d43072" (UID: "1c2c82b6-06c2-46a6-ae3d-cf3b40d43072"). InnerVolumeSpecName "kube-api-access-bb6s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:31:35.860104 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.860062    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c2c82b6-06c2-46a6-ae3d-cf3b40d43072" (UID: "1c2c82b6-06c2-46a6-ae3d-cf3b40d43072"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:31:35.959414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.959371    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:31:35.959414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.959413    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:31:35.959414 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:35.959424    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb6s7\" (UniqueName: \"kubernetes.io/projected/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072-kube-api-access-bb6s7\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:31:36.603963 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:36.603927    2584 generic.go:356] "Generic (PLEG): container finished" podID="1c2c82b6-06c2-46a6-ae3d-cf3b40d43072" containerID="e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762" exitCode=0
Jul 22 12:31:36.604400 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:36.603970    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"
Jul 22 12:31:36.604400 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:36.604010    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd" event={"ID":"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072","Type":"ContainerDied","Data":"e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762"}
Jul 22 12:31:36.604400 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:36.604065    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd" event={"ID":"1c2c82b6-06c2-46a6-ae3d-cf3b40d43072","Type":"ContainerDied","Data":"d428218c14d13784d0ffa471b5a0eff58421bc89231f60f7bfebdae4fe09611c"}
Jul 22 12:31:36.604400 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:36.604086    2584 scope.go:117] "RemoveContainer" containerID="e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762"
Jul 22 12:31:36.612840 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:36.612821    2584 scope.go:117] "RemoveContainer" containerID="e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762"
Jul 22 12:31:36.613080 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:31:36.613060    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762\": container with ID starting with e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762 not found: ID does not exist" containerID="e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762"
Jul 22 12:31:36.613145 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:36.613092    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762"} err="failed to get container status \"e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762\": rpc error: code = NotFound desc = could not find container \"e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762\": container with ID starting with e7924d8c21fd577f4194cff1e5b2ca5d4c4908a1c84286086f9960f7ca8d9762 not found: ID does not exist"
Jul 22 12:31:36.627779 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:36.627748    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"]
Jul 22 12:31:36.632549 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:36.632515    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a7ea4ba9-5c5d8fdb44-jjvsd"]
Jul 22 12:31:37.524860 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:31:37.524826    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c2c82b6-06c2-46a6-ae3d-cf3b40d43072" path="/var/lib/kubelet/pods/1c2c82b6-06c2-46a6-ae3d-cf3b40d43072/volumes"
Jul 22 12:33:30.362282 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.362246    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"]
Jul 22 12:33:30.362769 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.362624    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2c82b6-06c2-46a6-ae3d-cf3b40d43072" containerName="maas-api"
Jul 22 12:33:30.362769 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.362634    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2c82b6-06c2-46a6-ae3d-cf3b40d43072" containerName="maas-api"
Jul 22 12:33:30.362769 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.362708    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c2c82b6-06c2-46a6-ae3d-cf3b40d43072" containerName="maas-api"
Jul 22 12:33:30.362769 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.362718    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4b008d6-d377-4b75-9c77-50855ac1f3a6" containerName="istio-proxy"
Jul 22 12:33:30.365827 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.365811    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:30.368623 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.368597    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9ece34e1-serving-cert\""
Jul 22 12:33:30.368623 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.368604    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-svscs\""
Jul 22 12:33:30.375897 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.375874    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"]
Jul 22 12:33:30.404623 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.404593    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdk98\" (UniqueName: \"kubernetes.io/projected/0af27698-7548-46a8-b3d6-b9885214e34e-kube-api-access-tdk98\") pod \"maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:30.404790 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.404632    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af27698-7548-46a8-b3d6-b9885214e34e-maas-api-tls\") pod \"maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:30.404790 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.404658    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af27698-7548-46a8-b3d6-b9885214e34e-openshift-service-ca\") pod \"maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:30.505607 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.505574    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdk98\" (UniqueName: \"kubernetes.io/projected/0af27698-7548-46a8-b3d6-b9885214e34e-kube-api-access-tdk98\") pod \"maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:30.505768 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.505614    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af27698-7548-46a8-b3d6-b9885214e34e-maas-api-tls\") pod \"maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:30.505768 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.505640    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af27698-7548-46a8-b3d6-b9885214e34e-openshift-service-ca\") pod \"maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:30.506275 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.506258    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af27698-7548-46a8-b3d6-b9885214e34e-openshift-service-ca\") pod \"maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:30.507943 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.507927    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af27698-7548-46a8-b3d6-b9885214e34e-maas-api-tls\") pod \"maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:30.516080 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.516054    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdk98\" (UniqueName: \"kubernetes.io/projected/0af27698-7548-46a8-b3d6-b9885214e34e-kube-api-access-tdk98\") pod \"maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:30.677147 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.677064    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:30.821388 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:30.821348    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"]
Jul 22 12:33:30.822781 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:33:30.822747    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af27698_7548_46a8_b3d6_b9885214e34e.slice/crio-2cd434ccc163f2b6cb0e70d0089539bfdff9f5b4eaef7dd88df146fa5569bcf3 WatchSource:0}: Error finding container 2cd434ccc163f2b6cb0e70d0089539bfdff9f5b4eaef7dd88df146fa5569bcf3: Status 404 returned error can't find the container with id 2cd434ccc163f2b6cb0e70d0089539bfdff9f5b4eaef7dd88df146fa5569bcf3
Jul 22 12:33:31.136413 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:31.136354    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x" event={"ID":"0af27698-7548-46a8-b3d6-b9885214e34e","Type":"ContainerStarted","Data":"2cd434ccc163f2b6cb0e70d0089539bfdff9f5b4eaef7dd88df146fa5569bcf3"}
Jul 22 12:33:32.142901 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:32.142862    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x" event={"ID":"0af27698-7548-46a8-b3d6-b9885214e34e","Type":"ContainerStarted","Data":"e7210e740a432ef23d2a302e169c31de833724172228a360eecbd3754cd1f166"}
Jul 22 12:33:32.143302 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:32.142997    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:32.164535 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:32.164479    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x" podStartSLOduration=1.759487865 podStartE2EDuration="2.164458471s" totalImagesPullingTime="404.970606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:33:32.162687292 +0000 UTC m=+1901.184709011" watchObservedRunningTime="2026-07-22 12:33:32.164458471 +0000 UTC m=+1901.186480171"
Jul 22 12:33:38.154063 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:38.154030    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:53.080992 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.080960    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"]
Jul 22 12:33:53.081467 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.081155    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x" podUID="0af27698-7548-46a8-b3d6-b9885214e34e" containerName="maas-api" containerID="cri-o://e7210e740a432ef23d2a302e169c31de833724172228a360eecbd3754cd1f166" gracePeriod=30
Jul 22 12:33:53.148342 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.148296    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x" podUID="0af27698-7548-46a8-b3d6-b9885214e34e" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.48:8443/health\": dial tcp 10.134.0.48:8443: connect: connection refused"
Jul 22 12:33:53.248937 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.248903    2584 generic.go:356] "Generic (PLEG): container finished" podID="0af27698-7548-46a8-b3d6-b9885214e34e" containerID="e7210e740a432ef23d2a302e169c31de833724172228a360eecbd3754cd1f166" exitCode=0
Jul 22 12:33:53.249103 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.248981    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x" event={"ID":"0af27698-7548-46a8-b3d6-b9885214e34e","Type":"ContainerDied","Data":"e7210e740a432ef23d2a302e169c31de833724172228a360eecbd3754cd1f166"}
Jul 22 12:33:53.334425 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.334344    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:53.431863 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.431839    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af27698-7548-46a8-b3d6-b9885214e34e-openshift-service-ca\") pod \"0af27698-7548-46a8-b3d6-b9885214e34e\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") "
Jul 22 12:33:53.432016 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.431874    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af27698-7548-46a8-b3d6-b9885214e34e-maas-api-tls\") pod \"0af27698-7548-46a8-b3d6-b9885214e34e\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") "
Jul 22 12:33:53.432016 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.431908    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdk98\" (UniqueName: \"kubernetes.io/projected/0af27698-7548-46a8-b3d6-b9885214e34e-kube-api-access-tdk98\") pod \"0af27698-7548-46a8-b3d6-b9885214e34e\" (UID: \"0af27698-7548-46a8-b3d6-b9885214e34e\") "
Jul 22 12:33:53.432182 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.432157    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0af27698-7548-46a8-b3d6-b9885214e34e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0af27698-7548-46a8-b3d6-b9885214e34e" (UID: "0af27698-7548-46a8-b3d6-b9885214e34e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:33:53.433938 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.433908    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0af27698-7548-46a8-b3d6-b9885214e34e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0af27698-7548-46a8-b3d6-b9885214e34e" (UID: "0af27698-7548-46a8-b3d6-b9885214e34e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:33:53.434184 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.433948    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0af27698-7548-46a8-b3d6-b9885214e34e-kube-api-access-tdk98" (OuterVolumeSpecName: "kube-api-access-tdk98") pod "0af27698-7548-46a8-b3d6-b9885214e34e" (UID: "0af27698-7548-46a8-b3d6-b9885214e34e"). InnerVolumeSpecName "kube-api-access-tdk98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:33:53.533197 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.533164    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af27698-7548-46a8-b3d6-b9885214e34e-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:33:53.533342 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.533200    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af27698-7548-46a8-b3d6-b9885214e34e-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:33:53.533342 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:53.533222    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdk98\" (UniqueName: \"kubernetes.io/projected/0af27698-7548-46a8-b3d6-b9885214e34e-kube-api-access-tdk98\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:33:54.254819 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:54.254787    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x" event={"ID":"0af27698-7548-46a8-b3d6-b9885214e34e","Type":"ContainerDied","Data":"2cd434ccc163f2b6cb0e70d0089539bfdff9f5b4eaef7dd88df146fa5569bcf3"}
Jul 22 12:33:54.255271 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:54.254870    2584 scope.go:117] "RemoveContainer" containerID="e7210e740a432ef23d2a302e169c31de833724172228a360eecbd3754cd1f166"
Jul 22 12:33:54.255271 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:54.254801    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"
Jul 22 12:33:54.280029 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:54.280003    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"]
Jul 22 12:33:54.285960 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:54.285936    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ece34e1-686ccd456c-z8v4x"]
Jul 22 12:33:55.525654 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:33:55.525625    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0af27698-7548-46a8-b3d6-b9885214e34e" path="/var/lib/kubelet/pods/0af27698-7548-46a8-b3d6-b9885214e34e/volumes"
Jul 22 12:34:29.530071 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.530034    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"]
Jul 22 12:34:29.530656 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.530479    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0af27698-7548-46a8-b3d6-b9885214e34e" containerName="maas-api"
Jul 22 12:34:29.530656 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.530492    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0af27698-7548-46a8-b3d6-b9885214e34e" containerName="maas-api"
Jul 22 12:34:29.530656 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.530560    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0af27698-7548-46a8-b3d6-b9885214e34e" containerName="maas-api"
Jul 22 12:34:29.533884 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.533868    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:29.537510 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.537485    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a70c8883-serving-cert\""
Jul 22 12:34:29.537695 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.537675    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-svscs\""
Jul 22 12:34:29.549437 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.549408    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"]
Jul 22 12:34:29.675058 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.675021    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-maas-api-tls\") pod \"maas-api-e2e-mt-a70c8883-6846b6564-8b65j\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:29.675058 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.675061    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkgpn\" (UniqueName: \"kubernetes.io/projected/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-kube-api-access-tkgpn\") pod \"maas-api-e2e-mt-a70c8883-6846b6564-8b65j\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:29.675271 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.675151    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-openshift-service-ca\") pod \"maas-api-e2e-mt-a70c8883-6846b6564-8b65j\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:29.776012 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.775976    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-openshift-service-ca\") pod \"maas-api-e2e-mt-a70c8883-6846b6564-8b65j\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:29.776187 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.776127    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-maas-api-tls\") pod \"maas-api-e2e-mt-a70c8883-6846b6564-8b65j\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:29.776187 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.776148    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkgpn\" (UniqueName: \"kubernetes.io/projected/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-kube-api-access-tkgpn\") pod \"maas-api-e2e-mt-a70c8883-6846b6564-8b65j\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:29.776639 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.776616    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-openshift-service-ca\") pod \"maas-api-e2e-mt-a70c8883-6846b6564-8b65j\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:29.778593 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.778574    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-maas-api-tls\") pod \"maas-api-e2e-mt-a70c8883-6846b6564-8b65j\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:29.786238 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.786171    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkgpn\" (UniqueName: \"kubernetes.io/projected/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-kube-api-access-tkgpn\") pod \"maas-api-e2e-mt-a70c8883-6846b6564-8b65j\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:29.843943 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.843903    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:29.984112 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.984086    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"]
Jul 22 12:34:29.986478 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:34:29.986449    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c1a4cd6_a97e_4fb2_9d6a_71a8c79d2475.slice/crio-c5496cc1ccaba5d5ab4d2b590925db16e06b648734baf61be1642b1badc5912e WatchSource:0}: Error finding container c5496cc1ccaba5d5ab4d2b590925db16e06b648734baf61be1642b1badc5912e: Status 404 returned error can't find the container with id c5496cc1ccaba5d5ab4d2b590925db16e06b648734baf61be1642b1badc5912e
Jul 22 12:34:29.987835 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:29.987818    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:34:30.424803 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:30.424714    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j" event={"ID":"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475","Type":"ContainerStarted","Data":"c5496cc1ccaba5d5ab4d2b590925db16e06b648734baf61be1642b1badc5912e"}
Jul 22 12:34:31.430460 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:31.430418    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j" event={"ID":"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475","Type":"ContainerStarted","Data":"12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0"}
Jul 22 12:34:31.430870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:31.430553    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:34:31.452526 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:31.452478    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j" podStartSLOduration=2.0299351 podStartE2EDuration="2.452462219s" totalImagesPullingTime="422.527119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:34:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:34:31.450555038 +0000 UTC m=+1960.472576741" watchObservedRunningTime="2026-07-22 12:34:31.452462219 +0000 UTC m=+1960.474483976"
Jul 22 12:34:37.441852 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:34:37.441821    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:35:38.394109 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.394075    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"]
Jul 22 12:35:38.394553 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.394274    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j" podUID="8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475" containerName="maas-api" containerID="cri-o://12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0" gracePeriod=30
Jul 22 12:35:38.646234 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.646166    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:35:38.690433 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.690406    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-maas-api-tls\") pod \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") "
Jul 22 12:35:38.690603 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.690443    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-openshift-service-ca\") pod \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") "
Jul 22 12:35:38.690603 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.690528    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkgpn\" (UniqueName: \"kubernetes.io/projected/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-kube-api-access-tkgpn\") pod \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\" (UID: \"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475\") "
Jul 22 12:35:38.690853 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.690825    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475" (UID: "8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:35:38.692499 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.692467    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-kube-api-access-tkgpn" (OuterVolumeSpecName: "kube-api-access-tkgpn") pod "8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475" (UID: "8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475"). InnerVolumeSpecName "kube-api-access-tkgpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:35:38.692603 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.692541    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475" (UID: "8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:35:38.757738 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.757710    2584 generic.go:356] "Generic (PLEG): container finished" podID="8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475" containerID="12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0" exitCode=0
Jul 22 12:35:38.757870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.757759    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"
Jul 22 12:35:38.757870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.757765    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j" event={"ID":"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475","Type":"ContainerDied","Data":"12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0"}
Jul 22 12:35:38.757870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.757807    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j" event={"ID":"8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475","Type":"ContainerDied","Data":"c5496cc1ccaba5d5ab4d2b590925db16e06b648734baf61be1642b1badc5912e"}
Jul 22 12:35:38.757870 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.757827    2584 scope.go:117] "RemoveContainer" containerID="12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0"
Jul 22 12:35:38.766980 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.766963    2584 scope.go:117] "RemoveContainer" containerID="12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0"
Jul 22 12:35:38.767241 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:35:38.767225    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0\": container with ID starting with 12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0 not found: ID does not exist" containerID="12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0"
Jul 22 12:35:38.767293 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.767247    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0"} err="failed to get container status \"12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0\": rpc error: code = NotFound desc = could not find container \"12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0\": container with ID starting with 12fb3db7ead052439ebf28017c1bb2cadd3643519b7a2e0b2e1c45489fea73f0 not found: ID does not exist"
Jul 22 12:35:38.782585 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.782559    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"]
Jul 22 12:35:38.787196 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.787176    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a70c8883-6846b6564-8b65j"]
Jul 22 12:35:38.791922 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.791904    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkgpn\" (UniqueName: \"kubernetes.io/projected/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-kube-api-access-tkgpn\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:35:38.791986 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.791926    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:35:38.791986 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:38.791936    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:35:39.524961 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:35:39.524926    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475" path="/var/lib/kubelet/pods/8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475/volumes"
Jul 22 12:37:37.393931 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.393900    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"]
Jul 22 12:37:37.394489 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.394443    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475" containerName="maas-api"
Jul 22 12:37:37.394489 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.394471    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475" containerName="maas-api"
Jul 22 12:37:37.394641 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.394610    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c1a4cd6-a97e-4fb2-9d6a-71a8c79d2475" containerName="maas-api"
Jul 22 12:37:37.397750 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.397731    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:37.402159 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.401659    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-svscs\""
Jul 22 12:37:37.402159 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.401960    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-5e8367-serving-cert\""
Jul 22 12:37:37.410713 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.410604    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"]
Jul 22 12:37:37.469563 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.469531    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000e74c3-3ddc-4ca0-af60-31dbee21b828-maas-api-tls\") pod \"maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:37.469704 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.469631    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s46zp\" (UniqueName: \"kubernetes.io/projected/000e74c3-3ddc-4ca0-af60-31dbee21b828-kube-api-access-s46zp\") pod \"maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:37.469704 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.469656    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/000e74c3-3ddc-4ca0-af60-31dbee21b828-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:37.570905 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.570870    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/000e74c3-3ddc-4ca0-af60-31dbee21b828-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:37.571076 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.570926    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000e74c3-3ddc-4ca0-af60-31dbee21b828-maas-api-tls\") pod \"maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:37.571076 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.570999    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s46zp\" (UniqueName: \"kubernetes.io/projected/000e74c3-3ddc-4ca0-af60-31dbee21b828-kube-api-access-s46zp\") pod \"maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:37.571160 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:37:37.571087    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-serving-cert: secret "maas-api-e2e-inf-a-5e8367-serving-cert" not found
Jul 22 12:37:37.571198 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:37:37.571160    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000e74c3-3ddc-4ca0-af60-31dbee21b828-maas-api-tls podName:000e74c3-3ddc-4ca0-af60-31dbee21b828 nodeName:}" failed. No retries permitted until 2026-07-22 12:37:38.071144338 +0000 UTC m=+2147.093166017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/000e74c3-3ddc-4ca0-af60-31dbee21b828-maas-api-tls") pod "maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h" (UID: "000e74c3-3ddc-4ca0-af60-31dbee21b828") : secret "maas-api-e2e-inf-a-5e8367-serving-cert" not found
Jul 22 12:37:37.571532 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.571517    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/000e74c3-3ddc-4ca0-af60-31dbee21b828-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:37.580751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:37.580730    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s46zp\" (UniqueName: \"kubernetes.io/projected/000e74c3-3ddc-4ca0-af60-31dbee21b828-kube-api-access-s46zp\") pod \"maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:38.074851 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:38.074818    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000e74c3-3ddc-4ca0-af60-31dbee21b828-maas-api-tls\") pod \"maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:38.077251 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:38.077231    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000e74c3-3ddc-4ca0-af60-31dbee21b828-maas-api-tls\") pod \"maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:38.315272 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:38.315236    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:38.460761 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:38.460733    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"]
Jul 22 12:37:38.462784 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:37:38.462759    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000e74c3_3ddc_4ca0_af60_31dbee21b828.slice/crio-09a4217aa0834ac2c5eb1830b3d86439156bd52fcc0ea4a9f151f8d5a5aebe3f WatchSource:0}: Error finding container 09a4217aa0834ac2c5eb1830b3d86439156bd52fcc0ea4a9f151f8d5a5aebe3f: Status 404 returned error can't find the container with id 09a4217aa0834ac2c5eb1830b3d86439156bd52fcc0ea4a9f151f8d5a5aebe3f
Jul 22 12:37:39.324625 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:39.324594    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h" event={"ID":"000e74c3-3ddc-4ca0-af60-31dbee21b828","Type":"ContainerStarted","Data":"ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64"}
Jul 22 12:37:39.324796 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:39.324648    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h" event={"ID":"000e74c3-3ddc-4ca0-af60-31dbee21b828","Type":"ContainerStarted","Data":"09a4217aa0834ac2c5eb1830b3d86439156bd52fcc0ea4a9f151f8d5a5aebe3f"}
Jul 22 12:37:39.324796 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:39.324679    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:39.345102 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:39.345044    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h" podStartSLOduration=1.629960107 podStartE2EDuration="2.345028046s" totalImagesPullingTime="715.067939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:37:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:39.342891249 +0000 UTC m=+2148.364912979" watchObservedRunningTime="2026-07-22 12:37:39.345028046 +0000 UTC m=+2148.367049747"
Jul 22 12:37:45.334438 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.334355    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:37:45.410991 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.410954    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"]
Jul 22 12:37:45.414944 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.414920    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:45.417576 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.417546    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-07497a-serving-cert\""
Jul 22 12:37:45.425055 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.425028    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"]
Jul 22 12:37:45.548883 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.548848    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/593c4af0-b0e3-4d9e-8f19-f9011b53e665-openshift-service-ca\") pod \"maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:45.549049 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.548967    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nd9j\" (UniqueName: \"kubernetes.io/projected/593c4af0-b0e3-4d9e-8f19-f9011b53e665-kube-api-access-5nd9j\") pod \"maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:45.549049 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.549002    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/593c4af0-b0e3-4d9e-8f19-f9011b53e665-maas-api-tls\") pod \"maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:45.649947 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.649919    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/593c4af0-b0e3-4d9e-8f19-f9011b53e665-openshift-service-ca\") pod \"maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:45.650148 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.650008    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nd9j\" (UniqueName: \"kubernetes.io/projected/593c4af0-b0e3-4d9e-8f19-f9011b53e665-kube-api-access-5nd9j\") pod \"maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:45.650148 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.650043    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/593c4af0-b0e3-4d9e-8f19-f9011b53e665-maas-api-tls\") pod \"maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:45.650672 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.650648    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/593c4af0-b0e3-4d9e-8f19-f9011b53e665-openshift-service-ca\") pod \"maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:45.652294 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.652275    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/593c4af0-b0e3-4d9e-8f19-f9011b53e665-maas-api-tls\") pod \"maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:45.658941 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.658918    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nd9j\" (UniqueName: \"kubernetes.io/projected/593c4af0-b0e3-4d9e-8f19-f9011b53e665-kube-api-access-5nd9j\") pod \"maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:45.726890 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.726870    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:45.862500 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:45.862475    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"]
Jul 22 12:37:45.865251 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:37:45.865215    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod593c4af0_b0e3_4d9e_8f19_f9011b53e665.slice/crio-9843d2720fa18209b234a81c7d20f331bdb2042c5247d8b395fae2aa4ad1a5d5 WatchSource:0}: Error finding container 9843d2720fa18209b234a81c7d20f331bdb2042c5247d8b395fae2aa4ad1a5d5: Status 404 returned error can't find the container with id 9843d2720fa18209b234a81c7d20f331bdb2042c5247d8b395fae2aa4ad1a5d5
Jul 22 12:37:46.360113 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:46.360088    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn" event={"ID":"593c4af0-b0e3-4d9e-8f19-f9011b53e665","Type":"ContainerStarted","Data":"9843d2720fa18209b234a81c7d20f331bdb2042c5247d8b395fae2aa4ad1a5d5"}
Jul 22 12:37:47.366238 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:47.366207    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn" event={"ID":"593c4af0-b0e3-4d9e-8f19-f9011b53e665","Type":"ContainerStarted","Data":"2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548"}
Jul 22 12:37:47.366619 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:47.366274    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:37:47.398961 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:47.398896    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn" podStartSLOduration=1.9162397260000001 podStartE2EDuration="2.398881256s" totalImagesPullingTime="482.64153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:37:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:47.39757964 +0000 UTC m=+2156.419601342" watchObservedRunningTime="2026-07-22 12:37:47.398881256 +0000 UTC m=+2156.420902956"
Jul 22 12:37:53.376711 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:37:53.376681    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:38:30.702932 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.702900    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"]
Jul 22 12:38:30.706415 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.706395    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.710221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.710195    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-07497a\"/\"kube-root-ca.crt\""
Jul 22 12:38:30.710221 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.710219    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-07497a\"/\"openshift-service-ca.crt\""
Jul 22 12:38:30.710412 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.710192    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-07497a\"/\"default-dockercfg-jsbmf\""
Jul 22 12:38:30.710412 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.710197    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-07497a\"/\"test-model-07497a-kserve-self-signed-certs\""
Jul 22 12:38:30.715765 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.715742    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"]
Jul 22 12:38:30.769151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.769110    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-home\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.769151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.769155    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tmp-dir\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.769451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.769265    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lljgg\" (UniqueName: \"kubernetes.io/projected/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-kube-api-access-lljgg\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.769451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.769308    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-dshm\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.769451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.769351    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tls-certs\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.769451 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.769408    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-model-cache\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.870193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.870152    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lljgg\" (UniqueName: \"kubernetes.io/projected/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-kube-api-access-lljgg\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.870193 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.870191    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-dshm\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.870465 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.870211    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tls-certs\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.870465 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.870341    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-model-cache\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.870912 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.870885    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-model-cache\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.871184 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.871158    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-home\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.871279 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.870889    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-home\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.871279 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.871265    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tmp-dir\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.871603 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.871579    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tmp-dir\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.873074 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.873049    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tls-certs\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.873280 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.873261    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-dshm\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:30.882018 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:30.881985    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lljgg\" (UniqueName: \"kubernetes.io/projected/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-kube-api-access-lljgg\") pod \"test-model-07497a-kserve-6c98dffcd5-mzjzk\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") " pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:31.028662 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:31.028575    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:31.162903 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:31.162874    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"]
Jul 22 12:38:31.164893 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:38:31.164867    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5b3a024_bf9b_4f0a_aaaa_fb7961972dc5.slice/crio-3a5095f64e63543eed9951a6e87da125a73d1ca3b0828d3ffa8398fc49f9fb38 WatchSource:0}: Error finding container 3a5095f64e63543eed9951a6e87da125a73d1ca3b0828d3ffa8398fc49f9fb38: Status 404 returned error can't find the container with id 3a5095f64e63543eed9951a6e87da125a73d1ca3b0828d3ffa8398fc49f9fb38
Jul 22 12:38:31.575454 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:31.575414    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk" event={"ID":"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5","Type":"ContainerStarted","Data":"592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9"}
Jul 22 12:38:31.575454 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:31.575458    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk" event={"ID":"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5","Type":"ContainerStarted","Data":"3a5095f64e63543eed9951a6e87da125a73d1ca3b0828d3ffa8398fc49f9fb38"}
Jul 22 12:38:31.595104 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:31.595059    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk" podStartSLOduration=1.595044445 podStartE2EDuration="1.595044445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:38:31.593728506 +0000 UTC m=+2200.615750208" watchObservedRunningTime="2026-07-22 12:38:31.595044445 +0000 UTC m=+2200.617066145"
Jul 22 12:38:41.028885 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:41.028848    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:41.029320 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:41.028908    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:41.042850 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:41.042824    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:38:41.634804 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:38:41.634774    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:39:15.344337 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:15.344302    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"]
Jul 22 12:39:26.029456 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.029421    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"]
Jul 22 12:39:26.029829 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.029666    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h" podUID="000e74c3-3ddc-4ca0-af60-31dbee21b828" containerName="maas-api" containerID="cri-o://ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64" gracePeriod=30
Jul 22 12:39:26.283482 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.283429    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:39:26.302542 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.302477    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s46zp\" (UniqueName: \"kubernetes.io/projected/000e74c3-3ddc-4ca0-af60-31dbee21b828-kube-api-access-s46zp\") pod \"000e74c3-3ddc-4ca0-af60-31dbee21b828\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") "
Jul 22 12:39:26.302759 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.302727    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/000e74c3-3ddc-4ca0-af60-31dbee21b828-openshift-service-ca\") pod \"000e74c3-3ddc-4ca0-af60-31dbee21b828\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") "
Jul 22 12:39:26.302841 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.302788    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000e74c3-3ddc-4ca0-af60-31dbee21b828-maas-api-tls\") pod \"000e74c3-3ddc-4ca0-af60-31dbee21b828\" (UID: \"000e74c3-3ddc-4ca0-af60-31dbee21b828\") "
Jul 22 12:39:26.303255 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.303225    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/000e74c3-3ddc-4ca0-af60-31dbee21b828-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "000e74c3-3ddc-4ca0-af60-31dbee21b828" (UID: "000e74c3-3ddc-4ca0-af60-31dbee21b828"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:39:26.305313 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.305244    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/000e74c3-3ddc-4ca0-af60-31dbee21b828-kube-api-access-s46zp" (OuterVolumeSpecName: "kube-api-access-s46zp") pod "000e74c3-3ddc-4ca0-af60-31dbee21b828" (UID: "000e74c3-3ddc-4ca0-af60-31dbee21b828"). InnerVolumeSpecName "kube-api-access-s46zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:39:26.305457 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.305436    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/000e74c3-3ddc-4ca0-af60-31dbee21b828-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "000e74c3-3ddc-4ca0-af60-31dbee21b828" (UID: "000e74c3-3ddc-4ca0-af60-31dbee21b828"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:39:26.403647 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.403614    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s46zp\" (UniqueName: \"kubernetes.io/projected/000e74c3-3ddc-4ca0-af60-31dbee21b828-kube-api-access-s46zp\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:26.403647 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.403643    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/000e74c3-3ddc-4ca0-af60-31dbee21b828-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:26.403647 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.403652    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/000e74c3-3ddc-4ca0-af60-31dbee21b828-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:26.839879 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.839839    2584 generic.go:356] "Generic (PLEG): container finished" podID="000e74c3-3ddc-4ca0-af60-31dbee21b828" containerID="ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64" exitCode=0
Jul 22 12:39:26.839879 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.839878    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"
Jul 22 12:39:26.840097 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.839918    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h" event={"ID":"000e74c3-3ddc-4ca0-af60-31dbee21b828","Type":"ContainerDied","Data":"ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64"}
Jul 22 12:39:26.840097 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.839975    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h" event={"ID":"000e74c3-3ddc-4ca0-af60-31dbee21b828","Type":"ContainerDied","Data":"09a4217aa0834ac2c5eb1830b3d86439156bd52fcc0ea4a9f151f8d5a5aebe3f"}
Jul 22 12:39:26.840097 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.840003    2584 scope.go:117] "RemoveContainer" containerID="ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64"
Jul 22 12:39:26.849318 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.849302    2584 scope.go:117] "RemoveContainer" containerID="ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64"
Jul 22 12:39:26.849636 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:39:26.849613    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64\": container with ID starting with ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64 not found: ID does not exist" containerID="ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64"
Jul 22 12:39:26.849706 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.849637    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64"} err="failed to get container status \"ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64\": rpc error: code = NotFound desc = could not find container \"ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64\": container with ID starting with ed3e4bdf92bbfae2a41d7f6bd6f5143b6622ea16ab7f57797b57a7774dcdaa64 not found: ID does not exist"
Jul 22 12:39:26.863602 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.863577    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"]
Jul 22 12:39:26.868821 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:26.868801    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e8367-85b5b555bd-9lq9h"]
Jul 22 12:39:27.525519 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:27.525487    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="000e74c3-3ddc-4ca0-af60-31dbee21b828" path="/var/lib/kubelet/pods/000e74c3-3ddc-4ca0-af60-31dbee21b828/volumes"
Jul 22 12:39:30.365254 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.365207    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk" podUID="a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" containerName="main" containerID="cri-o://592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9" gracePeriod=45
Jul 22 12:39:30.622793 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.622732    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:39:30.640822 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.638458    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-dshm\") pod \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") "
Jul 22 12:39:30.640822 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.638520    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tls-certs\") pod \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") "
Jul 22 12:39:30.640822 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.638555    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-model-cache\") pod \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") "
Jul 22 12:39:30.640822 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.638602    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tmp-dir\") pod \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") "
Jul 22 12:39:30.640822 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.638785    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lljgg\" (UniqueName: \"kubernetes.io/projected/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-kube-api-access-lljgg\") pod \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") "
Jul 22 12:39:30.640822 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.638880    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-home\") pod \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\" (UID: \"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5\") "
Jul 22 12:39:30.641348 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.641297    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-home" (OuterVolumeSpecName: "home") pod "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" (UID: "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:30.641523 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.641397    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-model-cache" (OuterVolumeSpecName: "model-cache") pod "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" (UID: "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:30.642716 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.642692    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" (UID: "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:30.644059 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.644032    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-dshm" (OuterVolumeSpecName: "dshm") pod "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" (UID: "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:30.655792 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.653762    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-kube-api-access-lljgg" (OuterVolumeSpecName: "kube-api-access-lljgg") pod "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" (UID: "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5"). InnerVolumeSpecName "kube-api-access-lljgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:39:30.655792 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.655510    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" (UID: "a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:39:30.741136 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.741106    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lljgg\" (UniqueName: \"kubernetes.io/projected/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-kube-api-access-lljgg\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:30.741136 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.741132    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-home\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:30.741136 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.741141    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-dshm\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:30.741434 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.741149    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tls-certs\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:30.741434 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.741157    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-model-cache\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:30.741434 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.741164    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5-tmp-dir\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:30.862250 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.862213    2584 generic.go:356] "Generic (PLEG): container finished" podID="a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" containerID="592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9" exitCode=0
Jul 22 12:39:30.862427 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.862267    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"
Jul 22 12:39:30.862427 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.862292    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk" event={"ID":"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5","Type":"ContainerDied","Data":"592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9"}
Jul 22 12:39:30.862427 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.862332    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk" event={"ID":"a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5","Type":"ContainerDied","Data":"3a5095f64e63543eed9951a6e87da125a73d1ca3b0828d3ffa8398fc49f9fb38"}
Jul 22 12:39:30.862427 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.862351    2584 scope.go:117] "RemoveContainer" containerID="592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9"
Jul 22 12:39:30.872093 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.872073    2584 scope.go:117] "RemoveContainer" containerID="592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9"
Jul 22 12:39:30.872343 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:39:30.872325    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9\": container with ID starting with 592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9 not found: ID does not exist" containerID="592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9"
Jul 22 12:39:30.872430 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.872353    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9"} err="failed to get container status \"592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9\": rpc error: code = NotFound desc = could not find container \"592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9\": container with ID starting with 592aa66325a6bbea6e6f2d25fbb6b52e302a70b023968f650f8491692cdb47a9 not found: ID does not exist"
Jul 22 12:39:30.886094 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.886038    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"]
Jul 22 12:39:30.888036 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:30.888016    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-07497a/test-model-07497a-kserve-6c98dffcd5-mzjzk"]
Jul 22 12:39:31.525942 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:31.525900    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" path="/var/lib/kubelet/pods/a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5/volumes"
Jul 22 12:39:48.537057 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.537027    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"]
Jul 22 12:39:48.537628 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.537212    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn" podUID="593c4af0-b0e3-4d9e-8f19-f9011b53e665" containerName="maas-api" containerID="cri-o://2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548" gracePeriod=30
Jul 22 12:39:48.787678 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.787621    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:39:48.920364 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.920332    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/593c4af0-b0e3-4d9e-8f19-f9011b53e665-openshift-service-ca\") pod \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") "
Jul 22 12:39:48.920547 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.920418    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/593c4af0-b0e3-4d9e-8f19-f9011b53e665-maas-api-tls\") pod \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") "
Jul 22 12:39:48.920547 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.920458    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nd9j\" (UniqueName: \"kubernetes.io/projected/593c4af0-b0e3-4d9e-8f19-f9011b53e665-kube-api-access-5nd9j\") pod \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\" (UID: \"593c4af0-b0e3-4d9e-8f19-f9011b53e665\") "
Jul 22 12:39:48.920718 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.920693    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/593c4af0-b0e3-4d9e-8f19-f9011b53e665-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "593c4af0-b0e3-4d9e-8f19-f9011b53e665" (UID: "593c4af0-b0e3-4d9e-8f19-f9011b53e665"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:39:48.922496 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.922476    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/593c4af0-b0e3-4d9e-8f19-f9011b53e665-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "593c4af0-b0e3-4d9e-8f19-f9011b53e665" (UID: "593c4af0-b0e3-4d9e-8f19-f9011b53e665"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:39:48.922565 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.922495    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/593c4af0-b0e3-4d9e-8f19-f9011b53e665-kube-api-access-5nd9j" (OuterVolumeSpecName: "kube-api-access-5nd9j") pod "593c4af0-b0e3-4d9e-8f19-f9011b53e665" (UID: "593c4af0-b0e3-4d9e-8f19-f9011b53e665"). InnerVolumeSpecName "kube-api-access-5nd9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:39:48.944771 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.944741    2584 generic.go:356] "Generic (PLEG): container finished" podID="593c4af0-b0e3-4d9e-8f19-f9011b53e665" containerID="2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548" exitCode=0
Jul 22 12:39:48.944904 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.944780    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"
Jul 22 12:39:48.944904 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.944820    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn" event={"ID":"593c4af0-b0e3-4d9e-8f19-f9011b53e665","Type":"ContainerDied","Data":"2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548"}
Jul 22 12:39:48.944904 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.944877    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn" event={"ID":"593c4af0-b0e3-4d9e-8f19-f9011b53e665","Type":"ContainerDied","Data":"9843d2720fa18209b234a81c7d20f331bdb2042c5247d8b395fae2aa4ad1a5d5"}
Jul 22 12:39:48.944904 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.944900    2584 scope.go:117] "RemoveContainer" containerID="2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548"
Jul 22 12:39:48.953583 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.953566    2584 scope.go:117] "RemoveContainer" containerID="2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548"
Jul 22 12:39:48.953840 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:39:48.953814    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548\": container with ID starting with 2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548 not found: ID does not exist" containerID="2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548"
Jul 22 12:39:48.953910 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.953846    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548"} err="failed to get container status \"2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548\": rpc error: code = NotFound desc = could not find container \"2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548\": container with ID starting with 2a98b913247287580719b50e0b81d25c41508854314b9f78f0bdecb5d80d4548 not found: ID does not exist"
Jul 22 12:39:48.967257 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.967233    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"]
Jul 22 12:39:48.973898 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:48.973879    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-07497a-5c469f5f5c-r82dn"]
Jul 22 12:39:49.021541 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:49.021514    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/593c4af0-b0e3-4d9e-8f19-f9011b53e665-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:49.021541 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:49.021540    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/593c4af0-b0e3-4d9e-8f19-f9011b53e665-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:49.021687 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:49.021550    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nd9j\" (UniqueName: \"kubernetes.io/projected/593c4af0-b0e3-4d9e-8f19-f9011b53e665-kube-api-access-5nd9j\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:39:49.525199 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:39:49.525165    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="593c4af0-b0e3-4d9e-8f19-f9011b53e665" path="/var/lib/kubelet/pods/593c4af0-b0e3-4d9e-8f19-f9011b53e665/volumes"
Jul 22 12:40:00.933679 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.933649    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"]
Jul 22 12:40:00.934086 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.934046    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="000e74c3-3ddc-4ca0-af60-31dbee21b828" containerName="maas-api"
Jul 22 12:40:00.934086 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.934059    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="000e74c3-3ddc-4ca0-af60-31dbee21b828" containerName="maas-api"
Jul 22 12:40:00.934086 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.934068    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" containerName="main"
Jul 22 12:40:00.934086 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.934074    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" containerName="main"
Jul 22 12:40:00.934086 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.934084    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="593c4af0-b0e3-4d9e-8f19-f9011b53e665" containerName="maas-api"
Jul 22 12:40:00.934086 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.934090    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="593c4af0-b0e3-4d9e-8f19-f9011b53e665" containerName="maas-api"
Jul 22 12:40:00.934280 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.934157    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5b3a024-bf9b-4f0a-aaaa-fb7961972dc5" containerName="main"
Jul 22 12:40:00.934280 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.934167    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="593c4af0-b0e3-4d9e-8f19-f9011b53e665" containerName="maas-api"
Jul 22 12:40:00.934280 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.934175    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="000e74c3-3ddc-4ca0-af60-31dbee21b828" containerName="maas-api"
Jul 22 12:40:00.937454 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.937433    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:00.940137 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.940120    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-968672-openshift-default-dockercfg-2smj4\""
Jul 22 12:40:00.951440 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:00.951418    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"]
Jul 22 12:40:01.134504 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.134472    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-socket\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.134690 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.134512    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-certs\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.134690 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.134536    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-token\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.134690 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.134555    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcrm4\" (UniqueName: \"kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-kube-api-access-mcrm4\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.134690 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.134634    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-credential-socket\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.134690 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.134663    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-data\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.134859 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.134696    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-envoy\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.134859 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.134717    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istiod-ca-cert\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.134859 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.134754    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-podinfo\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.235353 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235269    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-socket\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.235353 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235305    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-certs\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.235353 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235324    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-token\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.235353 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235344    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcrm4\" (UniqueName: \"kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-kube-api-access-mcrm4\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.235679 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235408    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-credential-socket\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.235679 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235434    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-data\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.235679 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235477    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-envoy\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.235679 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235503    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istiod-ca-cert\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.235679 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235530    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-podinfo\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.235945 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235745    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-socket\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.235945 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235802    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-certs\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.236044 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235944    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-credential-socket\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.236044 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.235977    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-data\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.236209 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.236190    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istiod-ca-cert\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.237888 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.237859    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-envoy\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.238036 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.237967    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-podinfo\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.243901 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.243878    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-token\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.244225 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.244206    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcrm4\" (UniqueName: \"kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-kube-api-access-mcrm4\") pod \"e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") " pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.249967 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.249942    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:01.387831 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.387807    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"]
Jul 22 12:40:01.389585 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:40:01.389556    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c3cb60c_5cea_417e_8b4b_e3ca823d3269.slice/crio-d9e89f0cbd66645b0e426fc5d22d7daa94735df235f593b7674c7be1ea8a38a9 WatchSource:0}: Error finding container d9e89f0cbd66645b0e426fc5d22d7daa94735df235f593b7674c7be1ea8a38a9: Status 404 returned error can't find the container with id d9e89f0cbd66645b0e426fc5d22d7daa94735df235f593b7674c7be1ea8a38a9
Jul 22 12:40:01.391772 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.391742    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:01.391886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.391819    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:01.391945 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:01.391891    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:02.015011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:02.014972    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" event={"ID":"5c3cb60c-5cea-417e-8b4b-e3ca823d3269","Type":"ContainerStarted","Data":"d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53"}
Jul 22 12:40:02.015430 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:02.015063    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" event={"ID":"5c3cb60c-5cea-417e-8b4b-e3ca823d3269","Type":"ContainerStarted","Data":"d9e89f0cbd66645b0e426fc5d22d7daa94735df235f593b7674c7be1ea8a38a9"}
Jul 22 12:40:02.039117 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:02.039059    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" podStartSLOduration=2.039045457 podStartE2EDuration="2.039045457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:40:02.037188621 +0000 UTC m=+2291.059210326" watchObservedRunningTime="2026-07-22 12:40:02.039045457 +0000 UTC m=+2291.061067159"
Jul 22 12:40:02.250946 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:02.250917    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:02.256552 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:02.256524    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:03.020249 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:03.020216    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:03.021308 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:03.021292    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:04.041565 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.041534    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"]
Jul 22 12:40:04.045630 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.045610    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:04.048102 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.048085    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-968672-serving-cert\""
Jul 22 12:40:04.048102 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.048098    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-svscs\""
Jul 22 12:40:04.057473 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.057448    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"]
Jul 22 12:40:04.057799 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.057774    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqxf6\" (UniqueName: \"kubernetes.io/projected/daeb134b-f669-4588-930d-76597cd979ba-kube-api-access-fqxf6\") pod \"maas-api-e2e-api-a-968672-6565887cfc-v6rq7\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:04.057886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.057864    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daeb134b-f669-4588-930d-76597cd979ba-openshift-service-ca\") pod \"maas-api-e2e-api-a-968672-6565887cfc-v6rq7\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:04.057943 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.057925    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daeb134b-f669-4588-930d-76597cd979ba-maas-api-tls\") pod \"maas-api-e2e-api-a-968672-6565887cfc-v6rq7\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:04.158725 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.158689    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqxf6\" (UniqueName: \"kubernetes.io/projected/daeb134b-f669-4588-930d-76597cd979ba-kube-api-access-fqxf6\") pod \"maas-api-e2e-api-a-968672-6565887cfc-v6rq7\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:04.158892 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.158745    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daeb134b-f669-4588-930d-76597cd979ba-openshift-service-ca\") pod \"maas-api-e2e-api-a-968672-6565887cfc-v6rq7\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:04.158892 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.158803    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daeb134b-f669-4588-930d-76597cd979ba-maas-api-tls\") pod \"maas-api-e2e-api-a-968672-6565887cfc-v6rq7\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:04.159018 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:40:04.158909    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-968672-serving-cert: secret "maas-api-e2e-api-a-968672-serving-cert" not found
Jul 22 12:40:04.159018 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:40:04.158984    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daeb134b-f669-4588-930d-76597cd979ba-maas-api-tls podName:daeb134b-f669-4588-930d-76597cd979ba nodeName:}" failed. No retries permitted until 2026-07-22 12:40:04.658962793 +0000 UTC m=+2293.680984487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/daeb134b-f669-4588-930d-76597cd979ba-maas-api-tls") pod "maas-api-e2e-api-a-968672-6565887cfc-v6rq7" (UID: "daeb134b-f669-4588-930d-76597cd979ba") : secret "maas-api-e2e-api-a-968672-serving-cert" not found
Jul 22 12:40:04.159413 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.159393    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daeb134b-f669-4588-930d-76597cd979ba-openshift-service-ca\") pod \"maas-api-e2e-api-a-968672-6565887cfc-v6rq7\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:04.171095 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.171074    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqxf6\" (UniqueName: \"kubernetes.io/projected/daeb134b-f669-4588-930d-76597cd979ba-kube-api-access-fqxf6\") pod \"maas-api-e2e-api-a-968672-6565887cfc-v6rq7\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:04.664779 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.664735    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daeb134b-f669-4588-930d-76597cd979ba-maas-api-tls\") pod \"maas-api-e2e-api-a-968672-6565887cfc-v6rq7\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:04.667131 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.667112    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daeb134b-f669-4588-930d-76597cd979ba-maas-api-tls\") pod \"maas-api-e2e-api-a-968672-6565887cfc-v6rq7\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:04.957025 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:04.956939    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:05.310889 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:05.310865    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"]
Jul 22 12:40:05.312213 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:40:05.312188    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaeb134b_f669_4588_930d_76597cd979ba.slice/crio-0bda46da4ea44464c292f5dce08b200bed13357fea1e2e35e9c4f53c9fe59dd5 WatchSource:0}: Error finding container 0bda46da4ea44464c292f5dce08b200bed13357fea1e2e35e9c4f53c9fe59dd5: Status 404 returned error can't find the container with id 0bda46da4ea44464c292f5dce08b200bed13357fea1e2e35e9c4f53c9fe59dd5
Jul 22 12:40:05.313734 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:05.313718    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:40:06.038557 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:06.038524    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7" event={"ID":"daeb134b-f669-4588-930d-76597cd979ba","Type":"ContainerStarted","Data":"d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322"}
Jul 22 12:40:06.038738 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:06.038579    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7" event={"ID":"daeb134b-f669-4588-930d-76597cd979ba","Type":"ContainerStarted","Data":"0bda46da4ea44464c292f5dce08b200bed13357fea1e2e35e9c4f53c9fe59dd5"}
Jul 22 12:40:06.038738 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:06.038682    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:06.058182 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:06.058134    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7" podStartSLOduration=1.5341844770000002 podStartE2EDuration="2.058117858s" totalImagesPullingTime="523.933381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:40:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:40:06.057001184 +0000 UTC m=+2295.079022886" watchObservedRunningTime="2026-07-22 12:40:06.058117858 +0000 UTC m=+2295.080139559"
Jul 22 12:40:12.027666 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.027636    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"]
Jul 22 12:40:12.032225 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.032203    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:12.034664 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.034641    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-6749ef-serving-cert\""
Jul 22 12:40:12.047025 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.046994    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"]
Jul 22 12:40:12.080707 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.078738    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:12.151969 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.151925    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da3f300-e9e5-4048-8823-a54349eff73c-openshift-service-ca\") pod \"maas-api-e2e-api-b-6749ef-59bb7f685-fwprt\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:12.152281 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.152259    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ddgp\" (UniqueName: \"kubernetes.io/projected/1da3f300-e9e5-4048-8823-a54349eff73c-kube-api-access-7ddgp\") pod \"maas-api-e2e-api-b-6749ef-59bb7f685-fwprt\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:12.152566 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.152546    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1da3f300-e9e5-4048-8823-a54349eff73c-maas-api-tls\") pod \"maas-api-e2e-api-b-6749ef-59bb7f685-fwprt\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:12.256401 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.255871    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1da3f300-e9e5-4048-8823-a54349eff73c-maas-api-tls\") pod \"maas-api-e2e-api-b-6749ef-59bb7f685-fwprt\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:12.256401 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.255982    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da3f300-e9e5-4048-8823-a54349eff73c-openshift-service-ca\") pod \"maas-api-e2e-api-b-6749ef-59bb7f685-fwprt\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:12.256401 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.256020    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ddgp\" (UniqueName: \"kubernetes.io/projected/1da3f300-e9e5-4048-8823-a54349eff73c-kube-api-access-7ddgp\") pod \"maas-api-e2e-api-b-6749ef-59bb7f685-fwprt\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:12.256729 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:40:12.256475    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-serving-cert: secret "maas-api-e2e-api-b-6749ef-serving-cert" not found
Jul 22 12:40:12.256729 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:40:12.256556    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1da3f300-e9e5-4048-8823-a54349eff73c-maas-api-tls podName:1da3f300-e9e5-4048-8823-a54349eff73c nodeName:}" failed. No retries permitted until 2026-07-22 12:40:12.756533489 +0000 UTC m=+2301.778555170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1da3f300-e9e5-4048-8823-a54349eff73c-maas-api-tls") pod "maas-api-e2e-api-b-6749ef-59bb7f685-fwprt" (UID: "1da3f300-e9e5-4048-8823-a54349eff73c") : secret "maas-api-e2e-api-b-6749ef-serving-cert" not found
Jul 22 12:40:12.257405 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.257237    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da3f300-e9e5-4048-8823-a54349eff73c-openshift-service-ca\") pod \"maas-api-e2e-api-b-6749ef-59bb7f685-fwprt\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:12.267171 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.267151    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ddgp\" (UniqueName: \"kubernetes.io/projected/1da3f300-e9e5-4048-8823-a54349eff73c-kube-api-access-7ddgp\") pod \"maas-api-e2e-api-b-6749ef-59bb7f685-fwprt\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:12.760611 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.760548    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1da3f300-e9e5-4048-8823-a54349eff73c-maas-api-tls\") pod \"maas-api-e2e-api-b-6749ef-59bb7f685-fwprt\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:12.764192 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.764164    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1da3f300-e9e5-4048-8823-a54349eff73c-maas-api-tls\") pod \"maas-api-e2e-api-b-6749ef-59bb7f685-fwprt\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:12.982332 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:12.982296    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:13.170709 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:13.170677    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"]
Jul 22 12:40:13.171086 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:40:13.170883    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da3f300_e9e5_4048_8823_a54349eff73c.slice/crio-21e844ec8c4e70730cabc5a81fafb376f61514caa2953f08559bf9cb9c91ff8d WatchSource:0}: Error finding container 21e844ec8c4e70730cabc5a81fafb376f61514caa2953f08559bf9cb9c91ff8d: Status 404 returned error can't find the container with id 21e844ec8c4e70730cabc5a81fafb376f61514caa2953f08559bf9cb9c91ff8d
Jul 22 12:40:14.099357 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:14.099315    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt" event={"ID":"1da3f300-e9e5-4048-8823-a54349eff73c","Type":"ContainerStarted","Data":"c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b"}
Jul 22 12:40:14.099595 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:14.099396    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt" event={"ID":"1da3f300-e9e5-4048-8823-a54349eff73c","Type":"ContainerStarted","Data":"21e844ec8c4e70730cabc5a81fafb376f61514caa2953f08559bf9cb9c91ff8d"}
Jul 22 12:40:14.099595 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:14.099418    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:14.120540 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:14.120490    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt" podStartSLOduration=1.7550503339999999 podStartE2EDuration="2.120474259s" totalImagesPullingTime="365.423925ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:40:14.119904768 +0000 UTC m=+2303.141926483" watchObservedRunningTime="2026-07-22 12:40:14.120474259 +0000 UTC m=+2303.142495964"
Jul 22 12:40:18.022394 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:18.022349    2584 patch_prober.go:28] interesting pod/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 22 12:40:18.022897 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:18.022433    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 22 12:40:18.121730 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:18.121699    2584 generic.go:356] "Generic (PLEG): container finished" podID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerID="d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53" exitCode=0
Jul 22 12:40:18.121916 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:18.121771    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" event={"ID":"5c3cb60c-5cea-417e-8b4b-e3ca823d3269","Type":"ContainerDied","Data":"d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53"}
Jul 22 12:40:18.122249 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:18.122232    2584 scope.go:117] "RemoveContainer" containerID="d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53"
Jul 22 12:40:18.123335 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:18.123303    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:18.123432 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:18.123413    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:18.123475 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:18.123464    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:19.128151 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:19.128112    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" event={"ID":"5c3cb60c-5cea-417e-8b4b-e3ca823d3269","Type":"ContainerStarted","Data":"87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec"}
Jul 22 12:40:19.250472 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:19.250432    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:19.250675 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:19.250565    2584 patch_prober.go:28] interesting pod/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 22 12:40:19.250675 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:19.250613    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 22 12:40:20.110404 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:20.110356    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:40:20.250735 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:20.250700    2584 patch_prober.go:28] interesting pod/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 22 12:40:20.251311 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:20.251274    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 22 12:40:21.250833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:21.250795    2584 patch_prober.go:28] interesting pod/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 22 12:40:21.251457 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:21.251422    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 22 12:40:22.250601 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:22.250556    2584 patch_prober.go:28] interesting pod/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 22 12:40:22.250857 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:22.250625    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 22 12:40:23.255475 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:23.255443    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:23.255911 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:23.255823    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:23.256583 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:23.256566    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:40.272328 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.272287    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"]
Jul 22 12:40:40.272832 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.272561    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7" podUID="daeb134b-f669-4588-930d-76597cd979ba" containerName="maas-api" containerID="cri-o://d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322" gracePeriod=30
Jul 22 12:40:40.520748 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.520726    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:40.645955 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.645927    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daeb134b-f669-4588-930d-76597cd979ba-openshift-service-ca\") pod \"daeb134b-f669-4588-930d-76597cd979ba\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") "
Jul 22 12:40:40.646142 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.645997    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqxf6\" (UniqueName: \"kubernetes.io/projected/daeb134b-f669-4588-930d-76597cd979ba-kube-api-access-fqxf6\") pod \"daeb134b-f669-4588-930d-76597cd979ba\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") "
Jul 22 12:40:40.646142 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.646024    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daeb134b-f669-4588-930d-76597cd979ba-maas-api-tls\") pod \"daeb134b-f669-4588-930d-76597cd979ba\" (UID: \"daeb134b-f669-4588-930d-76597cd979ba\") "
Jul 22 12:40:40.646292 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.646270    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daeb134b-f669-4588-930d-76597cd979ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "daeb134b-f669-4588-930d-76597cd979ba" (UID: "daeb134b-f669-4588-930d-76597cd979ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:40:40.648143 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.648111    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/daeb134b-f669-4588-930d-76597cd979ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "daeb134b-f669-4588-930d-76597cd979ba" (UID: "daeb134b-f669-4588-930d-76597cd979ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:40:40.648339 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.648228    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daeb134b-f669-4588-930d-76597cd979ba-kube-api-access-fqxf6" (OuterVolumeSpecName: "kube-api-access-fqxf6") pod "daeb134b-f669-4588-930d-76597cd979ba" (UID: "daeb134b-f669-4588-930d-76597cd979ba"). InnerVolumeSpecName "kube-api-access-fqxf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:40:40.747233 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.747201    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daeb134b-f669-4588-930d-76597cd979ba-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:40.747233 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.747226    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqxf6\" (UniqueName: \"kubernetes.io/projected/daeb134b-f669-4588-930d-76597cd979ba-kube-api-access-fqxf6\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:40.747233 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:40.747237    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daeb134b-f669-4588-930d-76597cd979ba-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:41.237836 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:41.237804    2584 generic.go:356] "Generic (PLEG): container finished" podID="daeb134b-f669-4588-930d-76597cd979ba" containerID="d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322" exitCode=0
Jul 22 12:40:41.238062 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:41.237858    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"
Jul 22 12:40:41.238062 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:41.237864    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7" event={"ID":"daeb134b-f669-4588-930d-76597cd979ba","Type":"ContainerDied","Data":"d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322"}
Jul 22 12:40:41.238062 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:41.237911    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7" event={"ID":"daeb134b-f669-4588-930d-76597cd979ba","Type":"ContainerDied","Data":"0bda46da4ea44464c292f5dce08b200bed13357fea1e2e35e9c4f53c9fe59dd5"}
Jul 22 12:40:41.238062 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:41.237936    2584 scope.go:117] "RemoveContainer" containerID="d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322"
Jul 22 12:40:41.247593 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:41.247573    2584 scope.go:117] "RemoveContainer" containerID="d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322"
Jul 22 12:40:41.247846 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:40:41.247826    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322\": container with ID starting with d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322 not found: ID does not exist" containerID="d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322"
Jul 22 12:40:41.247909 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:41.247855    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322"} err="failed to get container status \"d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322\": rpc error: code = NotFound desc = could not find container \"d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322\": container with ID starting with d12f024607ec0a3a6c630aed1d20f9d5606435cde2c71277d30752526a852322 not found: ID does not exist"
Jul 22 12:40:41.260653 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:41.260626    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"]
Jul 22 12:40:41.265494 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:41.265470    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-968672-6565887cfc-v6rq7"]
Jul 22 12:40:41.528530 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:41.528448    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daeb134b-f669-4588-930d-76597cd979ba" path="/var/lib/kubelet/pods/daeb134b-f669-4588-930d-76597cd979ba/volumes"
Jul 22 12:40:51.670734 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:51.670650    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"]
Jul 22 12:40:51.671337 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:51.670867    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy" containerID="cri-o://87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec" gracePeriod=30
Jul 22 12:40:53.257875 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:53.257841    2584 patch_prober.go:28] interesting pod/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 12:40:53.258272 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:53.257904    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 12:40:56.928130 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:56.928106    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:57.004731 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.004634    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-socket\") pod \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") "
Jul 22 12:40:57.004731 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.004680    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-certs\") pod \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") "
Jul 22 12:40:57.004731 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.004709    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcrm4\" (UniqueName: \"kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-kube-api-access-mcrm4\") pod \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") "
Jul 22 12:40:57.005011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.004741    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-credential-socket\") pod \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") "
Jul 22 12:40:57.005011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.004789    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-data\") pod \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") "
Jul 22 12:40:57.005011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.004810    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-token\") pod \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") "
Jul 22 12:40:57.005011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.004852    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-envoy\") pod \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") "
Jul 22 12:40:57.005011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.004877    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-podinfo\") pod \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") "
Jul 22 12:40:57.005011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.004961    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5c3cb60c-5cea-417e-8b4b-e3ca823d3269" (UID: "5c3cb60c-5cea-417e-8b4b-e3ca823d3269"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:40:57.005011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.004987    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5c3cb60c-5cea-417e-8b4b-e3ca823d3269" (UID: "5c3cb60c-5cea-417e-8b4b-e3ca823d3269"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:40:57.005011 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.005011    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istiod-ca-cert\") pod \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\" (UID: \"5c3cb60c-5cea-417e-8b4b-e3ca823d3269\") "
Jul 22 12:40:57.005445 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.005220    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5c3cb60c-5cea-417e-8b4b-e3ca823d3269" (UID: "5c3cb60c-5cea-417e-8b4b-e3ca823d3269"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:40:57.005445 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.005372    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-socket\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:57.005445 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.005411    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-workload-certs\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:57.005445 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.005426    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-credential-socket\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:57.005661 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.005620    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5c3cb60c-5cea-417e-8b4b-e3ca823d3269" (UID: "5c3cb60c-5cea-417e-8b4b-e3ca823d3269"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:40:57.005751 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.005725    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-data" (OuterVolumeSpecName: "istio-data") pod "5c3cb60c-5cea-417e-8b4b-e3ca823d3269" (UID: "5c3cb60c-5cea-417e-8b4b-e3ca823d3269"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:40:57.007028 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.007006    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-kube-api-access-mcrm4" (OuterVolumeSpecName: "kube-api-access-mcrm4") pod "5c3cb60c-5cea-417e-8b4b-e3ca823d3269" (UID: "5c3cb60c-5cea-417e-8b4b-e3ca823d3269"). InnerVolumeSpecName "kube-api-access-mcrm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:40:57.007287 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.007270    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5c3cb60c-5cea-417e-8b4b-e3ca823d3269" (UID: "5c3cb60c-5cea-417e-8b4b-e3ca823d3269"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:40:57.007489 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.007469    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5c3cb60c-5cea-417e-8b4b-e3ca823d3269" (UID: "5c3cb60c-5cea-417e-8b4b-e3ca823d3269"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:40:57.007549 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.007488    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-token" (OuterVolumeSpecName: "istio-token") pod "5c3cb60c-5cea-417e-8b4b-e3ca823d3269" (UID: "5c3cb60c-5cea-417e-8b4b-e3ca823d3269"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:40:57.106820 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.106784    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcrm4\" (UniqueName: \"kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-kube-api-access-mcrm4\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:57.106820 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.106814    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-data\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:57.106820 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.106827    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-token\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:57.107071 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.106838    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-envoy\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:57.107071 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.106849    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istio-podinfo\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:57.107071 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.106859    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c3cb60c-5cea-417e-8b4b-e3ca823d3269-istiod-ca-cert\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:40:57.319146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.319047    2584 generic.go:356] "Generic (PLEG): container finished" podID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerID="87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec" exitCode=0
Jul 22 12:40:57.319146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.319101    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"
Jul 22 12:40:57.319146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.319116    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" event={"ID":"5c3cb60c-5cea-417e-8b4b-e3ca823d3269","Type":"ContainerDied","Data":"87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec"}
Jul 22 12:40:57.319471 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.319192    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz" event={"ID":"5c3cb60c-5cea-417e-8b4b-e3ca823d3269","Type":"ContainerDied","Data":"d9e89f0cbd66645b0e426fc5d22d7daa94735df235f593b7674c7be1ea8a38a9"}
Jul 22 12:40:57.319471 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.319217    2584 scope.go:117] "RemoveContainer" containerID="87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec"
Jul 22 12:40:57.328790 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.328763    2584 scope.go:117] "RemoveContainer" containerID="d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53"
Jul 22 12:40:57.336466 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.336448    2584 scope.go:117] "RemoveContainer" containerID="87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec"
Jul 22 12:40:57.336758 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:40:57.336742    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec\": container with ID starting with 87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec not found: ID does not exist" containerID="87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec"
Jul 22 12:40:57.336806 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.336765    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec"} err="failed to get container status \"87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec\": rpc error: code = NotFound desc = could not find container \"87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec\": container with ID starting with 87235591f80961b02efd83e51a3b27e5583b9535b9be0fd46a66d67a87b14aec not found: ID does not exist"
Jul 22 12:40:57.336806 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.336781    2584 scope.go:117] "RemoveContainer" containerID="d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53"
Jul 22 12:40:57.337042 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:40:57.337026    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53\": container with ID starting with d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53 not found: ID does not exist" containerID="d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53"
Jul 22 12:40:57.337088 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.337046    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53"} err="failed to get container status \"d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53\": rpc error: code = NotFound desc = could not find container \"d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53\": container with ID starting with d26315eff80a897a51fac61e003011232099b2c0d053436ce6a3a36fcb99ac53 not found: ID does not exist"
Jul 22 12:40:57.343717 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.343694    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"]
Jul 22 12:40:57.352617 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.352592    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-968672-openshift-default-7cc4d45579-hh9rz"]
Jul 22 12:40:57.525139 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:40:57.525099    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" path="/var/lib/kubelet/pods/5c3cb60c-5cea-417e-8b4b-e3ca823d3269/volumes"
Jul 22 12:41:07.504185 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.504148    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"]
Jul 22 12:41:07.504629 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.504441    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt" podUID="1da3f300-e9e5-4048-8823-a54349eff73c" containerName="maas-api" containerID="cri-o://c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b" gracePeriod=30
Jul 22 12:41:07.751134 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.751105    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:41:07.806372 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.806296    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1da3f300-e9e5-4048-8823-a54349eff73c-maas-api-tls\") pod \"1da3f300-e9e5-4048-8823-a54349eff73c\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") "
Jul 22 12:41:07.806372 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.806354    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da3f300-e9e5-4048-8823-a54349eff73c-openshift-service-ca\") pod \"1da3f300-e9e5-4048-8823-a54349eff73c\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") "
Jul 22 12:41:07.806600 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.806411    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ddgp\" (UniqueName: \"kubernetes.io/projected/1da3f300-e9e5-4048-8823-a54349eff73c-kube-api-access-7ddgp\") pod \"1da3f300-e9e5-4048-8823-a54349eff73c\" (UID: \"1da3f300-e9e5-4048-8823-a54349eff73c\") "
Jul 22 12:41:07.809028 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.806935    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da3f300-e9e5-4048-8823-a54349eff73c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1da3f300-e9e5-4048-8823-a54349eff73c" (UID: "1da3f300-e9e5-4048-8823-a54349eff73c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:41:07.811458 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.809597    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da3f300-e9e5-4048-8823-a54349eff73c-kube-api-access-7ddgp" (OuterVolumeSpecName: "kube-api-access-7ddgp") pod "1da3f300-e9e5-4048-8823-a54349eff73c" (UID: "1da3f300-e9e5-4048-8823-a54349eff73c"). InnerVolumeSpecName "kube-api-access-7ddgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:41:07.814239 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.814214    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da3f300-e9e5-4048-8823-a54349eff73c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1da3f300-e9e5-4048-8823-a54349eff73c" (UID: "1da3f300-e9e5-4048-8823-a54349eff73c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:41:07.908273 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.908243    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1da3f300-e9e5-4048-8823-a54349eff73c-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:41:07.908273 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.908272    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da3f300-e9e5-4048-8823-a54349eff73c-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:41:07.908273 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:07.908282    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ddgp\" (UniqueName: \"kubernetes.io/projected/1da3f300-e9e5-4048-8823-a54349eff73c-kube-api-access-7ddgp\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:41:08.374024 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:08.373982    2584 generic.go:356] "Generic (PLEG): container finished" podID="1da3f300-e9e5-4048-8823-a54349eff73c" containerID="c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b" exitCode=0
Jul 22 12:41:08.374211 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:08.374046    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"
Jul 22 12:41:08.374211 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:08.374071    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt" event={"ID":"1da3f300-e9e5-4048-8823-a54349eff73c","Type":"ContainerDied","Data":"c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b"}
Jul 22 12:41:08.374211 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:08.374150    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt" event={"ID":"1da3f300-e9e5-4048-8823-a54349eff73c","Type":"ContainerDied","Data":"21e844ec8c4e70730cabc5a81fafb376f61514caa2953f08559bf9cb9c91ff8d"}
Jul 22 12:41:08.374211 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:08.374171    2584 scope.go:117] "RemoveContainer" containerID="c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b"
Jul 22 12:41:08.383174 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:08.383158    2584 scope.go:117] "RemoveContainer" containerID="c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b"
Jul 22 12:41:08.383437 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:41:08.383411    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b\": container with ID starting with c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b not found: ID does not exist" containerID="c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b"
Jul 22 12:41:08.383521 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:08.383449    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b"} err="failed to get container status \"c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b\": rpc error: code = NotFound desc = could not find container \"c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b\": container with ID starting with c21f3c8444cc0908b5d46f499a3df39dc3e8e2fb4f0b55812706e656dd29866b not found: ID does not exist"
Jul 22 12:41:08.398723 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:08.398699    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"]
Jul 22 12:41:08.403222 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:08.403201    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6749ef-59bb7f685-fwprt"]
Jul 22 12:41:09.525029 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:09.524994    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1da3f300-e9e5-4048-8823-a54349eff73c" path="/var/lib/kubelet/pods/1da3f300-e9e5-4048-8823-a54349eff73c/volumes"
Jul 22 12:41:23.497752 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.497717    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"]
Jul 22 12:41:23.498287 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.498271    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy"
Jul 22 12:41:23.498340 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.498291    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy"
Jul 22 12:41:23.498340 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.498309    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da3f300-e9e5-4048-8823-a54349eff73c" containerName="maas-api"
Jul 22 12:41:23.498340 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.498317    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da3f300-e9e5-4048-8823-a54349eff73c" containerName="maas-api"
Jul 22 12:41:23.498483 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.498348    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daeb134b-f669-4588-930d-76597cd979ba" containerName="maas-api"
Jul 22 12:41:23.498483 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.498357    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="daeb134b-f669-4588-930d-76597cd979ba" containerName="maas-api"
Jul 22 12:41:23.498483 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.498454    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy"
Jul 22 12:41:23.498483 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.498466    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da3f300-e9e5-4048-8823-a54349eff73c" containerName="maas-api"
Jul 22 12:41:23.498483 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.498480    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="daeb134b-f669-4588-930d-76597cd979ba" containerName="maas-api"
Jul 22 12:41:23.503093 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.503074    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:23.505727 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.505708    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-a12988-serving-cert\""
Jul 22 12:41:23.505813 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.505766    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-svscs\""
Jul 22 12:41:23.512732 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.512706    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"]
Jul 22 12:41:23.550790 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.550763    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-maas-api-tls\") pod \"maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:23.550938 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.550801    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt2zz\" (UniqueName: \"kubernetes.io/projected/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-kube-api-access-pt2zz\") pod \"maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:23.550938 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.550831    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:23.651871 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.651837    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-maas-api-tls\") pod \"maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:23.652030 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.651880    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt2zz\" (UniqueName: \"kubernetes.io/projected/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-kube-api-access-pt2zz\") pod \"maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:23.652030 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.651922    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:23.652030 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:41:23.651993    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-serving-cert: secret "maas-api-e2e-shared-a-a12988-serving-cert" not found
Jul 22 12:41:23.652139 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:41:23.652068    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-maas-api-tls podName:0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c nodeName:}" failed. No retries permitted until 2026-07-22 12:41:24.15204641 +0000 UTC m=+2373.174068103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-maas-api-tls") pod "maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb" (UID: "0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c") : secret "maas-api-e2e-shared-a-a12988-serving-cert" not found
Jul 22 12:41:23.652632 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.652614    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:23.663513 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:23.663487    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt2zz\" (UniqueName: \"kubernetes.io/projected/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-kube-api-access-pt2zz\") pod \"maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:24.157579 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:24.157542    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-maas-api-tls\") pod \"maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:24.159904 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:24.159863    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-maas-api-tls\") pod \"maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:24.413824 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:24.413743    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:24.756728 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:24.756698    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"]
Jul 22 12:41:24.758368 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:41:24.758339    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb8cf98_ad2d_4ad7_a28b_cabeda040d3c.slice/crio-1a2184530310c71a513427603ca47352aaa84469c27495f54377d2e7256c3f32 WatchSource:0}: Error finding container 1a2184530310c71a513427603ca47352aaa84469c27495f54377d2e7256c3f32: Status 404 returned error can't find the container with id 1a2184530310c71a513427603ca47352aaa84469c27495f54377d2e7256c3f32
Jul 22 12:41:25.457815 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:25.457725    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb" event={"ID":"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c","Type":"ContainerStarted","Data":"08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c"}
Jul 22 12:41:25.457815 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:25.457784    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb" event={"ID":"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c","Type":"ContainerStarted","Data":"1a2184530310c71a513427603ca47352aaa84469c27495f54377d2e7256c3f32"}
Jul 22 12:41:25.457815 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:25.457809    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:25.476533 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:25.476477    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb" podStartSLOduration=2.082327033 podStartE2EDuration="2.476458469s" totalImagesPullingTime="394.131436ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:41:25.476153556 +0000 UTC m=+2374.498175257" watchObservedRunningTime="2026-07-22 12:41:25.476458469 +0000 UTC m=+2374.498480170"
Jul 22 12:41:31.468369 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:31.468341    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:41:32.305955 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.305922    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"]
Jul 22 12:41:32.306548 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.306532    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy"
Jul 22 12:41:32.306603 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.306552    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy"
Jul 22 12:41:32.306655 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.306644    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c3cb60c-5cea-417e-8b4b-e3ca823d3269" containerName="istio-proxy"
Jul 22 12:41:32.310040 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.310007    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:32.312890 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.312865    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-980f27-serving-cert\""
Jul 22 12:41:32.323639 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.323614    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"]
Jul 22 12:41:32.436989 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.436956    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d676l\" (UniqueName: \"kubernetes.io/projected/ee0b97aa-f763-4d40-8412-e54633e14547-kube-api-access-d676l\") pod \"maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:32.437191 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.437007    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee0b97aa-f763-4d40-8412-e54633e14547-openshift-service-ca\") pod \"maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:32.437191 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.437075    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee0b97aa-f763-4d40-8412-e54633e14547-maas-api-tls\") pod \"maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:32.538312 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.538283    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d676l\" (UniqueName: \"kubernetes.io/projected/ee0b97aa-f763-4d40-8412-e54633e14547-kube-api-access-d676l\") pod \"maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:32.538734 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.538327    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee0b97aa-f763-4d40-8412-e54633e14547-openshift-service-ca\") pod \"maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:32.538734 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.538457    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee0b97aa-f763-4d40-8412-e54633e14547-maas-api-tls\") pod \"maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:32.539191 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.539172    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee0b97aa-f763-4d40-8412-e54633e14547-openshift-service-ca\") pod \"maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:32.541079 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.541053    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee0b97aa-f763-4d40-8412-e54633e14547-maas-api-tls\") pod \"maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:32.548111 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.548083    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d676l\" (UniqueName: \"kubernetes.io/projected/ee0b97aa-f763-4d40-8412-e54633e14547-kube-api-access-d676l\") pod \"maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:32.620556 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.620532    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:32.756045 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:32.756023    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"]
Jul 22 12:41:32.758084 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:41:32.758056    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee0b97aa_f763_4d40_8412_e54633e14547.slice/crio-be08f7e42d027f6f3649a3bb85d88266e5e7e1e11e2747cf73e2366a8585a9c8 WatchSource:0}: Error finding container be08f7e42d027f6f3649a3bb85d88266e5e7e1e11e2747cf73e2366a8585a9c8: Status 404 returned error can't find the container with id be08f7e42d027f6f3649a3bb85d88266e5e7e1e11e2747cf73e2366a8585a9c8
Jul 22 12:41:33.500555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:33.500470    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk" event={"ID":"ee0b97aa-f763-4d40-8412-e54633e14547","Type":"ContainerStarted","Data":"3e71a0e5afc9e5e4b692d5b7e49c43fd366a9d4aaf07ea5bc85ca776ce376454"}
Jul 22 12:41:33.500555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:33.500522    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk" event={"ID":"ee0b97aa-f763-4d40-8412-e54633e14547","Type":"ContainerStarted","Data":"be08f7e42d027f6f3649a3bb85d88266e5e7e1e11e2747cf73e2366a8585a9c8"}
Jul 22 12:41:33.500739 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:33.500594    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:41:33.519134 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:33.519089    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk" podStartSLOduration=1.048578669 podStartE2EDuration="1.51907281s" totalImagesPullingTime="470.494141ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:41:33.517632077 +0000 UTC m=+2382.539653805" watchObservedRunningTime="2026-07-22 12:41:33.51907281 +0000 UTC m=+2382.541094511"
Jul 22 12:41:39.511835 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:41:39.511806    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:42:28.305408 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.305360    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"]
Jul 22 12:42:28.308960 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.308943    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.311514 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.311488    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"openshift-service-ca.crt\""
Jul 22 12:42:28.312617 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.312592    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"auth-test-model-980f27-kserve-self-signed-certs\""
Jul 22 12:42:28.312722 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.312621    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"default-dockercfg-9q558\""
Jul 22 12:42:28.312722 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.312630    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"kube-root-ca.crt\""
Jul 22 12:42:28.319820 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.319796    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"]
Jul 22 12:42:28.450664 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.450625    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-dshm\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.450910 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.450723    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tls-certs\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.450910 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.450754    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-home\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.450910 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.450781    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tmp-dir\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.450910 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.450804    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jj6z\" (UniqueName: \"kubernetes.io/projected/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-kube-api-access-7jj6z\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.451092 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.450939    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-model-cache\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.551665 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.551634    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tls-certs\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.551665 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.551674    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-home\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.551917 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.551698    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tmp-dir\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.551917 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.551721    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jj6z\" (UniqueName: \"kubernetes.io/projected/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-kube-api-access-7jj6z\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.551917 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.551801    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-model-cache\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.552082 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.551987    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-dshm\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.552142 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.552090    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-home\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.552208 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.552166    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-model-cache\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.552288 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.552270    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tmp-dir\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.554084 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.554061    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-dshm\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.554257 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.554240    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tls-certs\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.560814 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.560761    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jj6z\" (UniqueName: \"kubernetes.io/projected/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-kube-api-access-7jj6z\") pod \"auth-test-model-980f27-kserve-8fbc898bc-c27xg\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") " pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.629941 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.629911    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:28.768911 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:28.768861    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"]
Jul 22 12:42:28.771210 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:42:28.771183    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b2d179e_588d_4f6b_a2e2_d76f280d6d21.slice/crio-22658aba15d36bd5b9e002d5aa7a1d53ee45dac93f4a01213264da27b775426b WatchSource:0}: Error finding container 22658aba15d36bd5b9e002d5aa7a1d53ee45dac93f4a01213264da27b775426b: Status 404 returned error can't find the container with id 22658aba15d36bd5b9e002d5aa7a1d53ee45dac93f4a01213264da27b775426b
Jul 22 12:42:29.768443 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:29.768402    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg" event={"ID":"1b2d179e-588d-4f6b-a2e2-d76f280d6d21","Type":"ContainerStarted","Data":"353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188"}
Jul 22 12:42:29.768443 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:29.768450    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg" event={"ID":"1b2d179e-588d-4f6b-a2e2-d76f280d6d21","Type":"ContainerStarted","Data":"22658aba15d36bd5b9e002d5aa7a1d53ee45dac93f4a01213264da27b775426b"}
Jul 22 12:42:29.788550 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:29.788405    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg" podStartSLOduration=1.7883540070000001 podStartE2EDuration="1.788354007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:42:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:42:29.787249008 +0000 UTC m=+2438.809270744" watchObservedRunningTime="2026-07-22 12:42:29.788354007 +0000 UTC m=+2438.810375709"
Jul 22 12:42:38.630357 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:38.630321    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:38.630814 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:38.630373    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:38.643304 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:38.643279    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:42:38.820886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:42:38.820856    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:44:32.996030 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:32.995996    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"]
Jul 22 12:44:48.016742 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.016679    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg" podUID="1b2d179e-588d-4f6b-a2e2-d76f280d6d21" containerName="main" containerID="cri-o://353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188" gracePeriod=45
Jul 22 12:44:48.271782 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.271719    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:44:48.381288 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.381258    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tmp-dir\") pod \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") "
Jul 22 12:44:48.381474 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.381301    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jj6z\" (UniqueName: \"kubernetes.io/projected/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-kube-api-access-7jj6z\") pod \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") "
Jul 22 12:44:48.381474 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.381326    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-home\") pod \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") "
Jul 22 12:44:48.381474 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.381362    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-dshm\") pod \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") "
Jul 22 12:44:48.381643 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.381473    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tls-certs\") pod \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") "
Jul 22 12:44:48.381643 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.381501    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-model-cache\") pod \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\" (UID: \"1b2d179e-588d-4f6b-a2e2-d76f280d6d21\") "
Jul 22 12:44:48.381643 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.381561    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-home" (OuterVolumeSpecName: "home") pod "1b2d179e-588d-4f6b-a2e2-d76f280d6d21" (UID: "1b2d179e-588d-4f6b-a2e2-d76f280d6d21"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:44:48.381643 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.381590    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1b2d179e-588d-4f6b-a2e2-d76f280d6d21" (UID: "1b2d179e-588d-4f6b-a2e2-d76f280d6d21"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:44:48.381854 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.381780    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-model-cache" (OuterVolumeSpecName: "model-cache") pod "1b2d179e-588d-4f6b-a2e2-d76f280d6d21" (UID: "1b2d179e-588d-4f6b-a2e2-d76f280d6d21"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:44:48.381854 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.381821    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tmp-dir\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:44:48.381854 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.381836    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-home\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:44:48.383708 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.383674    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1b2d179e-588d-4f6b-a2e2-d76f280d6d21" (UID: "1b2d179e-588d-4f6b-a2e2-d76f280d6d21"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:44:48.383708 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.383674    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-dshm" (OuterVolumeSpecName: "dshm") pod "1b2d179e-588d-4f6b-a2e2-d76f280d6d21" (UID: "1b2d179e-588d-4f6b-a2e2-d76f280d6d21"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:44:48.383857 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.383738    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-kube-api-access-7jj6z" (OuterVolumeSpecName: "kube-api-access-7jj6z") pod "1b2d179e-588d-4f6b-a2e2-d76f280d6d21" (UID: "1b2d179e-588d-4f6b-a2e2-d76f280d6d21"). InnerVolumeSpecName "kube-api-access-7jj6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:44:48.424316 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.424282    2584 generic.go:356] "Generic (PLEG): container finished" podID="1b2d179e-588d-4f6b-a2e2-d76f280d6d21" containerID="353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188" exitCode=0
Jul 22 12:44:48.424466 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.424324    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"
Jul 22 12:44:48.424466 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.424350    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg" event={"ID":"1b2d179e-588d-4f6b-a2e2-d76f280d6d21","Type":"ContainerDied","Data":"353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188"}
Jul 22 12:44:48.424466 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.424404    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg" event={"ID":"1b2d179e-588d-4f6b-a2e2-d76f280d6d21","Type":"ContainerDied","Data":"22658aba15d36bd5b9e002d5aa7a1d53ee45dac93f4a01213264da27b775426b"}
Jul 22 12:44:48.424466 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.424421    2584 scope.go:117] "RemoveContainer" containerID="353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188"
Jul 22 12:44:48.433244 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.433228    2584 scope.go:117] "RemoveContainer" containerID="353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188"
Jul 22 12:44:48.433537 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:44:48.433514    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188\": container with ID starting with 353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188 not found: ID does not exist" containerID="353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188"
Jul 22 12:44:48.433616 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.433547    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188"} err="failed to get container status \"353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188\": rpc error: code = NotFound desc = could not find container \"353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188\": container with ID starting with 353fde5215e86e13bae004a95e7b51dff632136145cb2eafd90144803a442188 not found: ID does not exist"
Jul 22 12:44:48.447532 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.447504    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"]
Jul 22 12:44:48.452895 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.452875    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/auth-test-model-980f27-kserve-8fbc898bc-c27xg"]
Jul 22 12:44:48.482613 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.482591    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-tls-certs\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:44:48.482613 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.482614    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-model-cache\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:44:48.482754 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.482624    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jj6z\" (UniqueName: \"kubernetes.io/projected/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-kube-api-access-7jj6z\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:44:48.482754 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:48.482632    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b2d179e-588d-4f6b-a2e2-d76f280d6d21-dshm\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:44:49.526894 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:44:49.526860    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b2d179e-588d-4f6b-a2e2-d76f280d6d21" path="/var/lib/kubelet/pods/1b2d179e-588d-4f6b-a2e2-d76f280d6d21/volumes"
Jul 22 12:45:09.687825 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.687786    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"]
Jul 22 12:45:09.688466 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.688338    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b2d179e-588d-4f6b-a2e2-d76f280d6d21" containerName="main"
Jul 22 12:45:09.688466 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.688358    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b2d179e-588d-4f6b-a2e2-d76f280d6d21" containerName="main"
Jul 22 12:45:09.688601 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.688503    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b2d179e-588d-4f6b-a2e2-d76f280d6d21" containerName="main"
Jul 22 12:45:09.691689 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.691667    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.694405 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.694359    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"openshift-service-ca.crt\""
Jul 22 12:45:09.695605 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.695587    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"default-dockercfg-9q558\""
Jul 22 12:45:09.695687 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.695623    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"kube-root-ca.crt\""
Jul 22 12:45:09.695687 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.695624    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"sub-test-model-980f27-kserve-self-signed-certs\""
Jul 22 12:45:09.700690 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.700668    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"]
Jul 22 12:45:09.782435 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.782399    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4k8j\" (UniqueName: \"kubernetes.io/projected/1c47c730-a7f1-417b-a59a-1e4f679d457a-kube-api-access-d4k8j\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.782597 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.782465    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-home\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.782597 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.782495    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-tmp-dir\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.782597 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.782529    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-dshm\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.782768 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.782599    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c47c730-a7f1-417b-a59a-1e4f679d457a-tls-certs\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.782768 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.782655    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-model-cache\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.883140 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.883104    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4k8j\" (UniqueName: \"kubernetes.io/projected/1c47c730-a7f1-417b-a59a-1e4f679d457a-kube-api-access-d4k8j\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.883286 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.883156    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-home\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.883286 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.883183    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-tmp-dir\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.883286 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.883211    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-dshm\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.883286 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.883236    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c47c730-a7f1-417b-a59a-1e4f679d457a-tls-certs\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.883286 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.883267    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-model-cache\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.883674 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.883649    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-home\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.883731 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.883660    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-tmp-dir\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.883731 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.883722    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-model-cache\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.885597 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.885567    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-dshm\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.885807 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.885789    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c47c730-a7f1-417b-a59a-1e4f679d457a-tls-certs\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:09.891946 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:09.891925    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4k8j\" (UniqueName: \"kubernetes.io/projected/1c47c730-a7f1-417b-a59a-1e4f679d457a-kube-api-access-d4k8j\") pod \"sub-test-model-980f27-kserve-54fc989ddc-zqdzx\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") " pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:10.013178 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:10.013090    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:10.148305 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:10.148278    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"]
Jul 22 12:45:10.150863 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:45:10.150823    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c47c730_a7f1_417b_a59a_1e4f679d457a.slice/crio-b0b75f2bd7ad7df42ad76dbeb942f951229add3fdfaa8df465b1b88504056465 WatchSource:0}: Error finding container b0b75f2bd7ad7df42ad76dbeb942f951229add3fdfaa8df465b1b88504056465: Status 404 returned error can't find the container with id b0b75f2bd7ad7df42ad76dbeb942f951229add3fdfaa8df465b1b88504056465
Jul 22 12:45:10.531047 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:10.531013    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx" event={"ID":"1c47c730-a7f1-417b-a59a-1e4f679d457a","Type":"ContainerStarted","Data":"8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e"}
Jul 22 12:45:10.531047 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:10.531050    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx" event={"ID":"1c47c730-a7f1-417b-a59a-1e4f679d457a","Type":"ContainerStarted","Data":"b0b75f2bd7ad7df42ad76dbeb942f951229add3fdfaa8df465b1b88504056465"}
Jul 22 12:45:10.551068 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:10.551024    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx" podStartSLOduration=1.5510108219999998 podStartE2EDuration="1.551010822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:45:10.549574289 +0000 UTC m=+2599.571596016" watchObservedRunningTime="2026-07-22 12:45:10.551010822 +0000 UTC m=+2599.573032524"
Jul 22 12:45:20.013503 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:20.013452    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:20.013503 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:20.013519    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:20.026253 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:20.026230    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:20.589896 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:20.589868    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:45:56.284893 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:45:56.284856    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"]
Jul 22 12:46:11.304842 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.304774    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx" podUID="1c47c730-a7f1-417b-a59a-1e4f679d457a" containerName="main" containerID="cri-o://8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e" gracePeriod=45
Jul 22 12:46:11.565913 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.565850    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:46:11.681162 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.681135    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-dshm\") pod \"1c47c730-a7f1-417b-a59a-1e4f679d457a\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") "
Jul 22 12:46:11.681324 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.681169    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4k8j\" (UniqueName: \"kubernetes.io/projected/1c47c730-a7f1-417b-a59a-1e4f679d457a-kube-api-access-d4k8j\") pod \"1c47c730-a7f1-417b-a59a-1e4f679d457a\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") "
Jul 22 12:46:11.681324 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.681250    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c47c730-a7f1-417b-a59a-1e4f679d457a-tls-certs\") pod \"1c47c730-a7f1-417b-a59a-1e4f679d457a\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") "
Jul 22 12:46:11.681324 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.681297    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-tmp-dir\") pod \"1c47c730-a7f1-417b-a59a-1e4f679d457a\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") "
Jul 22 12:46:11.681324 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.681314    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-home\") pod \"1c47c730-a7f1-417b-a59a-1e4f679d457a\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") "
Jul 22 12:46:11.681555 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.681366    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-model-cache\") pod \"1c47c730-a7f1-417b-a59a-1e4f679d457a\" (UID: \"1c47c730-a7f1-417b-a59a-1e4f679d457a\") "
Jul 22 12:46:11.681620 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.681582    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-home" (OuterVolumeSpecName: "home") pod "1c47c730-a7f1-417b-a59a-1e4f679d457a" (UID: "1c47c730-a7f1-417b-a59a-1e4f679d457a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:46:11.681620 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.681593    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1c47c730-a7f1-417b-a59a-1e4f679d457a" (UID: "1c47c730-a7f1-417b-a59a-1e4f679d457a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:46:11.681738 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.681687    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-model-cache" (OuterVolumeSpecName: "model-cache") pod "1c47c730-a7f1-417b-a59a-1e4f679d457a" (UID: "1c47c730-a7f1-417b-a59a-1e4f679d457a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:46:11.681795 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.681737    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-tmp-dir\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:46:11.681795 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.681758    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-home\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:46:11.683201 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.683176    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-dshm" (OuterVolumeSpecName: "dshm") pod "1c47c730-a7f1-417b-a59a-1e4f679d457a" (UID: "1c47c730-a7f1-417b-a59a-1e4f679d457a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:46:11.683322 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.683261    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c47c730-a7f1-417b-a59a-1e4f679d457a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1c47c730-a7f1-417b-a59a-1e4f679d457a" (UID: "1c47c730-a7f1-417b-a59a-1e4f679d457a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:46:11.683369 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.683354    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c47c730-a7f1-417b-a59a-1e4f679d457a-kube-api-access-d4k8j" (OuterVolumeSpecName: "kube-api-access-d4k8j") pod "1c47c730-a7f1-417b-a59a-1e4f679d457a" (UID: "1c47c730-a7f1-417b-a59a-1e4f679d457a"). InnerVolumeSpecName "kube-api-access-d4k8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:46:11.782355 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.782319    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-dshm\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:46:11.782355 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.782349    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4k8j\" (UniqueName: \"kubernetes.io/projected/1c47c730-a7f1-417b-a59a-1e4f679d457a-kube-api-access-d4k8j\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:46:11.782355 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.782360    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c47c730-a7f1-417b-a59a-1e4f679d457a-tls-certs\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:46:11.782619 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.782369    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c47c730-a7f1-417b-a59a-1e4f679d457a-model-cache\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:46:11.826516 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.826415    2584 generic.go:356] "Generic (PLEG): container finished" podID="1c47c730-a7f1-417b-a59a-1e4f679d457a" containerID="8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e" exitCode=0
Jul 22 12:46:11.826516 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.826451    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"
Jul 22 12:46:11.826516 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.826482    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx" event={"ID":"1c47c730-a7f1-417b-a59a-1e4f679d457a","Type":"ContainerDied","Data":"8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e"}
Jul 22 12:46:11.826826 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.826525    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx" event={"ID":"1c47c730-a7f1-417b-a59a-1e4f679d457a","Type":"ContainerDied","Data":"b0b75f2bd7ad7df42ad76dbeb942f951229add3fdfaa8df465b1b88504056465"}
Jul 22 12:46:11.826826 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.826542    2584 scope.go:117] "RemoveContainer" containerID="8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e"
Jul 22 12:46:11.835311 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.835295    2584 scope.go:117] "RemoveContainer" containerID="8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e"
Jul 22 12:46:11.835573 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:46:11.835555    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e\": container with ID starting with 8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e not found: ID does not exist" containerID="8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e"
Jul 22 12:46:11.835641 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.835580    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e"} err="failed to get container status \"8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e\": rpc error: code = NotFound desc = could not find container \"8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e\": container with ID starting with 8a0fe7ba77be47dea5ec6ae1c097911c9423cccf2887cfd4ed52e988943f6a8e not found: ID does not exist"
Jul 22 12:46:11.849886 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.849862    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"]
Jul 22 12:46:11.858094 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:11.858072    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/sub-test-model-980f27-kserve-54fc989ddc-zqdzx"]
Jul 22 12:46:13.525207 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:13.525178    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c47c730-a7f1-417b-a59a-1e4f679d457a" path="/var/lib/kubelet/pods/1c47c730-a7f1-417b-a59a-1e4f679d457a/volumes"
Jul 22 12:46:32.551041 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.551009    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"]
Jul 22 12:46:32.551537 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.551431    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c47c730-a7f1-417b-a59a-1e4f679d457a" containerName="main"
Jul 22 12:46:32.551537 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.551445    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c47c730-a7f1-417b-a59a-1e4f679d457a" containerName="main"
Jul 22 12:46:32.551537 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.551533    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c47c730-a7f1-417b-a59a-1e4f679d457a" containerName="main"
Jul 22 12:46:32.554623 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.554603    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.558617 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.558594    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"rate-test-model-980f27-kserve-self-signed-certs\""
Jul 22 12:46:32.558831 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.558626    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"default-dockercfg-9q558\""
Jul 22 12:46:32.559115 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.558692    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"kube-root-ca.crt\""
Jul 22 12:46:32.559115 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.558702    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-980f27\"/\"openshift-service-ca.crt\""
Jul 22 12:46:32.579768 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.579710    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"]
Jul 22 12:46:32.669981 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.669953    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-dshm\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.669981 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.669998    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx94f\" (UniqueName: \"kubernetes.io/projected/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-kube-api-access-fx94f\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.670240 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.670027    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tmp-dir\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.670240 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.670104    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tls-certs\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.670240 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.670146    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-model-cache\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.670240 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.670173    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-home\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.771752 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.771722    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-dshm\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.771943 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.771778    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx94f\" (UniqueName: \"kubernetes.io/projected/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-kube-api-access-fx94f\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.771943 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.771824    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tmp-dir\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.771943 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.771853    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tls-certs\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.771943 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.771918    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-model-cache\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.772173 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.771962    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-home\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.772235 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.772213    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tmp-dir\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.772368 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.772340    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-model-cache\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.772465 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.772364    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-home\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.773968 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.773946    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-dshm\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.774329 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.774310    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tls-certs\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.783126 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.783102    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx94f\" (UniqueName: \"kubernetes.io/projected/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-kube-api-access-fx94f\") pod \"rate-test-model-980f27-kserve-67dc89b6c5-fc48f\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") " pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:32.872018 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:32.871989    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:33.014416 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:33.014366    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"]
Jul 22 12:46:33.016019 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:46:33.015991    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab13f5ff_6534_4e2a_8b4a_d3313ced59bb.slice/crio-ca3692a49e86ae53f36fd346a4266d6821dda54b717cac19aa853a8b2538c54f WatchSource:0}: Error finding container ca3692a49e86ae53f36fd346a4266d6821dda54b717cac19aa853a8b2538c54f: Status 404 returned error can't find the container with id ca3692a49e86ae53f36fd346a4266d6821dda54b717cac19aa853a8b2538c54f
Jul 22 12:46:33.934069 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:33.934035    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f" event={"ID":"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb","Type":"ContainerStarted","Data":"9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33"}
Jul 22 12:46:33.934464 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:33.934080    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f" event={"ID":"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb","Type":"ContainerStarted","Data":"ca3692a49e86ae53f36fd346a4266d6821dda54b717cac19aa853a8b2538c54f"}
Jul 22 12:46:33.955084 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:33.955035    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f" podStartSLOduration=1.9550199209999999 podStartE2EDuration="1.955019921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:46:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:46:33.954291839 +0000 UTC m=+2682.976313564" watchObservedRunningTime="2026-07-22 12:46:33.955019921 +0000 UTC m=+2682.977041663"
Jul 22 12:46:42.872664 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:42.872565    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:42.872664 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:42.872619    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:42.885142 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:42.885119    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:46:42.987925 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:46:42.987898    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:47:16.518310 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:16.518278    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"]
Jul 22 12:47:31.538615 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.538571    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f" podUID="ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" containerName="main" containerID="cri-o://9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33" gracePeriod=45
Jul 22 12:47:31.796218 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.796157    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:47:31.938329 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.938293    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-model-cache\") pod \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") "
Jul 22 12:47:31.938546 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.938371    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-dshm\") pod \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") "
Jul 22 12:47:31.938546 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.938421    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fx94f\" (UniqueName: \"kubernetes.io/projected/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-kube-api-access-fx94f\") pod \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") "
Jul 22 12:47:31.938546 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.938476    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tmp-dir\") pod \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") "
Jul 22 12:47:31.938546 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.938510    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tls-certs\") pod \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") "
Jul 22 12:47:31.938770 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.938550    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-home\") pod \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\" (UID: \"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb\") "
Jul 22 12:47:31.938770 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.938650    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" (UID: "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:47:31.938906 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.938885    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-home" (OuterVolumeSpecName: "home") pod "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" (UID: "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:47:31.938978 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.938958    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-model-cache\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:47:31.939024 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.938996    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" (UID: "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:47:31.940650 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.940619    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-kube-api-access-fx94f" (OuterVolumeSpecName: "kube-api-access-fx94f") pod "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" (UID: "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb"). InnerVolumeSpecName "kube-api-access-fx94f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:47:31.940650 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.940643    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-dshm" (OuterVolumeSpecName: "dshm") pod "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" (UID: "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:47:31.940801 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:31.940744    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" (UID: "ab13f5ff-6534-4e2a-8b4a-d3313ced59bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:47:32.040027 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.039991    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-dshm\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:47:32.040027 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.040021    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fx94f\" (UniqueName: \"kubernetes.io/projected/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-kube-api-access-fx94f\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:47:32.040027 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.040031    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tmp-dir\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:47:32.040281 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.040040    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-tls-certs\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:47:32.040281 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.040047    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb-home\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:47:32.214925 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.214890    2584 generic.go:356] "Generic (PLEG): container finished" podID="ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" containerID="9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33" exitCode=0
Jul 22 12:47:32.215097 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.214929    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"
Jul 22 12:47:32.215097 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.214951    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f" event={"ID":"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb","Type":"ContainerDied","Data":"9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33"}
Jul 22 12:47:32.215097 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.214984    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f" event={"ID":"ab13f5ff-6534-4e2a-8b4a-d3313ced59bb","Type":"ContainerDied","Data":"ca3692a49e86ae53f36fd346a4266d6821dda54b717cac19aa853a8b2538c54f"}
Jul 22 12:47:32.215097 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.215005    2584 scope.go:117] "RemoveContainer" containerID="9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33"
Jul 22 12:47:32.224700 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.224684    2584 scope.go:117] "RemoveContainer" containerID="9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33"
Jul 22 12:47:32.224961 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:47:32.224942    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33\": container with ID starting with 9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33 not found: ID does not exist" containerID="9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33"
Jul 22 12:47:32.225041 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.224967    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33"} err="failed to get container status \"9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33\": rpc error: code = NotFound desc = could not find container \"9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33\": container with ID starting with 9ef22c87212536b9c0776fdb80657d6e7b0626fe73582d167e05dea93659df33 not found: ID does not exist"
Jul 22 12:47:32.243034 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.243003    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"]
Jul 22 12:47:32.248823 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:32.248801    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-980f27/rate-test-model-980f27-kserve-67dc89b6c5-fc48f"]
Jul 22 12:47:33.526145 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:33.526109    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" path="/var/lib/kubelet/pods/ab13f5ff-6534-4e2a-8b4a-d3313ced59bb/volumes"
Jul 22 12:47:45.971127 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:45.971093    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"]
Jul 22 12:47:45.971537 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:45.971292    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb" podUID="0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c" containerName="maas-api" containerID="cri-o://08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c" gracePeriod=30
Jul 22 12:47:46.222115 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.222055    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:47:46.278004 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.277961    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt2zz\" (UniqueName: \"kubernetes.io/projected/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-kube-api-access-pt2zz\") pod \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") "
Jul 22 12:47:46.278167 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.278052    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-maas-api-tls\") pod \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") "
Jul 22 12:47:46.278167 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.278081    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-openshift-service-ca\") pod \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\" (UID: \"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c\") "
Jul 22 12:47:46.278477 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.278453    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c" (UID: "0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:47:46.280161 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.280139    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c" (UID: "0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:47:46.280214 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.280198    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-kube-api-access-pt2zz" (OuterVolumeSpecName: "kube-api-access-pt2zz") pod "0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c" (UID: "0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c"). InnerVolumeSpecName "kube-api-access-pt2zz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:47:46.293550 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.293523    2584 generic.go:356] "Generic (PLEG): container finished" podID="0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c" containerID="08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c" exitCode=0
Jul 22 12:47:46.293683 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.293576    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"
Jul 22 12:47:46.293683 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.293610    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb" event={"ID":"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c","Type":"ContainerDied","Data":"08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c"}
Jul 22 12:47:46.293761 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.293713    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb" event={"ID":"0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c","Type":"ContainerDied","Data":"1a2184530310c71a513427603ca47352aaa84469c27495f54377d2e7256c3f32"}
Jul 22 12:47:46.293761 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.293744    2584 scope.go:117] "RemoveContainer" containerID="08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c"
Jul 22 12:47:46.302654 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.302638    2584 scope.go:117] "RemoveContainer" containerID="08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c"
Jul 22 12:47:46.302901 ip-10-0-143-107 kubenswrapper[2584]: E0722 12:47:46.302883    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c\": container with ID starting with 08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c not found: ID does not exist" containerID="08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c"
Jul 22 12:47:46.302967 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.302914    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c"} err="failed to get container status \"08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c\": rpc error: code = NotFound desc = could not find container \"08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c\": container with ID starting with 08bdde8480489ce6aa6c32b192d599ae75d58fddea7e29c1a262e4bef2c8607c not found: ID does not exist"
Jul 22 12:47:46.317921 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.317897    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"]
Jul 22 12:47:46.322106 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.322082    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a12988-699bfcd9d5-6jgvb"]
Jul 22 12:47:46.379050 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.379013    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:47:46.379050 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.379045    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:47:46.379234 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:46.379059    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt2zz\" (UniqueName: \"kubernetes.io/projected/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c-kube-api-access-pt2zz\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:47:47.525290 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:47:47.525251    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c" path="/var/lib/kubelet/pods/0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c/volumes"
Jul 22 12:48:10.235437 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.235404    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"]
Jul 22 12:48:10.236024 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.235622    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk" podUID="ee0b97aa-f763-4d40-8412-e54633e14547" containerName="maas-api" containerID="cri-o://3e71a0e5afc9e5e4b692d5b7e49c43fd366a9d4aaf07ea5bc85ca776ce376454" gracePeriod=30
Jul 22 12:48:10.408264 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.408234    2584 generic.go:356] "Generic (PLEG): container finished" podID="ee0b97aa-f763-4d40-8412-e54633e14547" containerID="3e71a0e5afc9e5e4b692d5b7e49c43fd366a9d4aaf07ea5bc85ca776ce376454" exitCode=0
Jul 22 12:48:10.408405 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.408308    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk" event={"ID":"ee0b97aa-f763-4d40-8412-e54633e14547","Type":"ContainerDied","Data":"3e71a0e5afc9e5e4b692d5b7e49c43fd366a9d4aaf07ea5bc85ca776ce376454"}
Jul 22 12:48:10.484868 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.484847    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:48:10.616150 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.616125    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee0b97aa-f763-4d40-8412-e54633e14547-maas-api-tls\") pod \"ee0b97aa-f763-4d40-8412-e54633e14547\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") "
Jul 22 12:48:10.616318 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.616185    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee0b97aa-f763-4d40-8412-e54633e14547-openshift-service-ca\") pod \"ee0b97aa-f763-4d40-8412-e54633e14547\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") "
Jul 22 12:48:10.616318 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.616210    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d676l\" (UniqueName: \"kubernetes.io/projected/ee0b97aa-f763-4d40-8412-e54633e14547-kube-api-access-d676l\") pod \"ee0b97aa-f763-4d40-8412-e54633e14547\" (UID: \"ee0b97aa-f763-4d40-8412-e54633e14547\") "
Jul 22 12:48:10.616606 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.616580    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee0b97aa-f763-4d40-8412-e54633e14547-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee0b97aa-f763-4d40-8412-e54633e14547" (UID: "ee0b97aa-f763-4d40-8412-e54633e14547"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:48:10.618138 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.618118    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee0b97aa-f763-4d40-8412-e54633e14547-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee0b97aa-f763-4d40-8412-e54633e14547" (UID: "ee0b97aa-f763-4d40-8412-e54633e14547"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:48:10.618245 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.618226    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee0b97aa-f763-4d40-8412-e54633e14547-kube-api-access-d676l" (OuterVolumeSpecName: "kube-api-access-d676l") pod "ee0b97aa-f763-4d40-8412-e54633e14547" (UID: "ee0b97aa-f763-4d40-8412-e54633e14547"). InnerVolumeSpecName "kube-api-access-d676l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:48:10.717146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.717119    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d676l\" (UniqueName: \"kubernetes.io/projected/ee0b97aa-f763-4d40-8412-e54633e14547-kube-api-access-d676l\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:48:10.717146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.717144    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee0b97aa-f763-4d40-8412-e54633e14547-maas-api-tls\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:48:10.717146 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:10.717153    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee0b97aa-f763-4d40-8412-e54633e14547-openshift-service-ca\") on node \"ip-10-0-143-107.ec2.internal\" DevicePath \"\""
Jul 22 12:48:11.414313 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:11.414282    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"
Jul 22 12:48:11.414313 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:11.414289    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk" event={"ID":"ee0b97aa-f763-4d40-8412-e54633e14547","Type":"ContainerDied","Data":"be08f7e42d027f6f3649a3bb85d88266e5e7e1e11e2747cf73e2366a8585a9c8"}
Jul 22 12:48:11.414804 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:11.414350    2584 scope.go:117] "RemoveContainer" containerID="3e71a0e5afc9e5e4b692d5b7e49c43fd366a9d4aaf07ea5bc85ca776ce376454"
Jul 22 12:48:11.438556 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:11.438528    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"]
Jul 22 12:48:11.442969 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:11.442945    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-980f27-74c8676d49-mbsjk"]
Jul 22 12:48:11.525636 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:11.525594    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee0b97aa-f763-4d40-8412-e54633e14547" path="/var/lib/kubelet/pods/ee0b97aa-f763-4d40-8412-e54633e14547/volumes"
Jul 22 12:48:28.967236 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:28.967208    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58d867cfcf-zxv9g_8b94f66f-f022-414b-8784-603142645fde/manager/0.log"
Jul 22 12:48:29.445342 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:29.445314    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-979495d87-d4lcn_96084ba3-4eb5-4380-ba10-6e2bffc98431/copy-manifests/0.log"
Jul 22 12:48:29.452707 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:29.452684    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-979495d87-d4lcn_96084ba3-4eb5-4380-ba10-6e2bffc98431/manager/0.log"
Jul 22 12:48:29.932124 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:29.932097    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7bd5fc67c4-x6rwt_f91fa26b-8859-4de9-a73d-48f2550d32e6/maas-api/0.log"
Jul 22 12:48:30.400069 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:30.400036    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-m72j4_f3ce581b-e7c0-41ad-8dcd-32f90b319e60/postgres/0.log"
Jul 22 12:48:31.803130 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:31.803095    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6zwcq_b79c82ae-1508-4ae7-b3ab-1bff041d62a5/manager/0.log"
Jul 22 12:48:31.917776 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:31.917746    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-74ctw_d3aa1483-f29a-4b34-8803-6937fc93280a/manager/0.log"
Jul 22 12:48:32.141410 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:32.141364    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mmsrn_fccd99f4-0482-4da0-9bdb-5c337818cf78/registry-server/0.log"
Jul 22 12:48:32.261621 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:32.261593    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-nvc27_6c961749-41f2-4f28-b0a1-cc672cb297a2/manager/0.log"
Jul 22 12:48:33.344988 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:33.344957    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fhvks_e5c60d29-9b5e-4b24-a0ac-4e8d118c2270/istio-proxy/1.log"
Jul 22 12:48:33.692745 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:33.692721    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d58b7fcb-9b8kf_62903732-d37c-4b43-b094-963c0910ac8b/router/0.log"
Jul 22 12:48:34.031986 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:34.031909    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-8jqm5_d4d00e16-c225-4ed0-b8ba-139e4eca75f8/main/0.log"
Jul 22 12:48:34.261601 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:34.261572    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-56hxt_ea714d9e-fd4a-42f5-8f5d-1d2d4406a859/main/0.log"
Jul 22 12:48:34.503366 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:34.503338    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-scvlf_0801a305-ec36-4442-91a1-43a99ef3ad7a/main/0.log"
Jul 22 12:48:39.830810 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.830775    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9s8cv/must-gather-pxzff"]
Jul 22 12:48:39.831185 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.831164    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" containerName="main"
Jul 22 12:48:39.831185 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.831175    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" containerName="main"
Jul 22 12:48:39.831263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.831195    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c" containerName="maas-api"
Jul 22 12:48:39.831263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.831201    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c" containerName="maas-api"
Jul 22 12:48:39.831263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.831213    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee0b97aa-f763-4d40-8412-e54633e14547" containerName="maas-api"
Jul 22 12:48:39.831263 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.831223    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee0b97aa-f763-4d40-8412-e54633e14547" containerName="maas-api"
Jul 22 12:48:39.831412 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.831278    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb8cf98-ad2d-4ad7-a28b-cabeda040d3c" containerName="maas-api"
Jul 22 12:48:39.831412 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.831286    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee0b97aa-f763-4d40-8412-e54633e14547" containerName="maas-api"
Jul 22 12:48:39.831412 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.831297    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab13f5ff-6534-4e2a-8b4a-d3313ced59bb" containerName="main"
Jul 22 12:48:39.834362 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.834345    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9s8cv/must-gather-pxzff"
Jul 22 12:48:39.836899 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.836871    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9s8cv\"/\"openshift-service-ca.crt\""
Jul 22 12:48:39.837002 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.836956    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9s8cv\"/\"default-dockercfg-nssjf\""
Jul 22 12:48:39.837057 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.837013    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9s8cv\"/\"kube-root-ca.crt\""
Jul 22 12:48:39.848653 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.848627    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9s8cv/must-gather-pxzff"]
Jul 22 12:48:39.990856 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.990815    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfnjg\" (UniqueName: \"kubernetes.io/projected/e46fa5ab-8858-40bd-8144-fa0318b3687e-kube-api-access-hfnjg\") pod \"must-gather-pxzff\" (UID: \"e46fa5ab-8858-40bd-8144-fa0318b3687e\") " pod="openshift-must-gather-9s8cv/must-gather-pxzff"
Jul 22 12:48:39.991023 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:39.990865    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e46fa5ab-8858-40bd-8144-fa0318b3687e-must-gather-output\") pod \"must-gather-pxzff\" (UID: \"e46fa5ab-8858-40bd-8144-fa0318b3687e\") " pod="openshift-must-gather-9s8cv/must-gather-pxzff"
Jul 22 12:48:40.092244 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:40.092157    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfnjg\" (UniqueName: \"kubernetes.io/projected/e46fa5ab-8858-40bd-8144-fa0318b3687e-kube-api-access-hfnjg\") pod \"must-gather-pxzff\" (UID: \"e46fa5ab-8858-40bd-8144-fa0318b3687e\") " pod="openshift-must-gather-9s8cv/must-gather-pxzff"
Jul 22 12:48:40.092244 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:40.092203    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e46fa5ab-8858-40bd-8144-fa0318b3687e-must-gather-output\") pod \"must-gather-pxzff\" (UID: \"e46fa5ab-8858-40bd-8144-fa0318b3687e\") " pod="openshift-must-gather-9s8cv/must-gather-pxzff"
Jul 22 12:48:40.092596 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:40.092577    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e46fa5ab-8858-40bd-8144-fa0318b3687e-must-gather-output\") pod \"must-gather-pxzff\" (UID: \"e46fa5ab-8858-40bd-8144-fa0318b3687e\") " pod="openshift-must-gather-9s8cv/must-gather-pxzff"
Jul 22 12:48:40.104180 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:40.104150    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfnjg\" (UniqueName: \"kubernetes.io/projected/e46fa5ab-8858-40bd-8144-fa0318b3687e-kube-api-access-hfnjg\") pod \"must-gather-pxzff\" (UID: \"e46fa5ab-8858-40bd-8144-fa0318b3687e\") " pod="openshift-must-gather-9s8cv/must-gather-pxzff"
Jul 22 12:48:40.163947 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:40.163918    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9s8cv/must-gather-pxzff"
Jul 22 12:48:40.302010 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:40.301985    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9s8cv/must-gather-pxzff"]
Jul 22 12:48:40.303785 ip-10-0-143-107 kubenswrapper[2584]: W0722 12:48:40.303758    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode46fa5ab_8858_40bd_8144_fa0318b3687e.slice/crio-fe6e4b1395fc08b06c2eacf3146f63708afbcc53128b5eeb11af596f4f9a788f WatchSource:0}: Error finding container fe6e4b1395fc08b06c2eacf3146f63708afbcc53128b5eeb11af596f4f9a788f: Status 404 returned error can't find the container with id fe6e4b1395fc08b06c2eacf3146f63708afbcc53128b5eeb11af596f4f9a788f
Jul 22 12:48:40.305610 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:40.305593    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:48:40.549904 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:40.549867    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s8cv/must-gather-pxzff" event={"ID":"e46fa5ab-8858-40bd-8144-fa0318b3687e","Type":"ContainerStarted","Data":"fe6e4b1395fc08b06c2eacf3146f63708afbcc53128b5eeb11af596f4f9a788f"}
Jul 22 12:48:41.558938 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:41.558841    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s8cv/must-gather-pxzff" event={"ID":"e46fa5ab-8858-40bd-8144-fa0318b3687e","Type":"ContainerStarted","Data":"6f8a6ebeccb830d0d9306ae00e926c6298876ecf418be03cc00c886730505aec"}
Jul 22 12:48:41.558938 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:41.558898    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s8cv/must-gather-pxzff" event={"ID":"e46fa5ab-8858-40bd-8144-fa0318b3687e","Type":"ContainerStarted","Data":"0d437bec8c1be503a267a29c6524fe0e769c6ee92967efdc3e8d72db393a1759"}
Jul 22 12:48:41.579544 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:41.579499    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9s8cv/must-gather-pxzff" podStartSLOduration=1.746109776 podStartE2EDuration="2.57948258s" totalImagesPullingTime="833.372804ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:48:41.579208259 +0000 UTC m=+2810.601229982" watchObservedRunningTime="2026-07-22 12:48:41.57948258 +0000 UTC m=+2810.601504300"
Jul 22 12:48:42.798586 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:42.798554    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kds22_f78d8d30-61ed-4179-a3d4-bd7188b22aab/global-pull-secret-syncer/0.log"
Jul 22 12:48:43.092638 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:43.092567    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pmljl_464a2ed7-bcdb-4426-b488-9d826b166405/konnectivity-agent/0.log"
Jul 22 12:48:43.222261 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:43.222229    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-107.ec2.internal_acd362fd4cb61d67b387c384f06f0cd7/haproxy/0.log"
Jul 22 12:48:48.463217 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:48.462582    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6zwcq_b79c82ae-1508-4ae7-b3ab-1bff041d62a5/manager/0.log"
Jul 22 12:48:48.487333 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:48.487245    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-74ctw_d3aa1483-f29a-4b34-8803-6937fc93280a/manager/0.log"
Jul 22 12:48:48.564330 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:48.564192    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mmsrn_fccd99f4-0482-4da0-9bdb-5c337818cf78/registry-server/0.log"
Jul 22 12:48:48.748255 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:48.748181    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-nvc27_6c961749-41f2-4f28-b0a1-cc672cb297a2/manager/0.log"
Jul 22 12:48:50.756003 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:50.755969    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-cxxpr_59d9f112-d3ab-47da-8873-d13561388b67/cluster-monitoring-operator/0.log"
Jul 22 12:48:50.791934 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:50.791906    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-lk7j8_40c7a4cf-1267-4e3d-ad93-c0d94078b181/kube-state-metrics/0.log"
Jul 22 12:48:50.822544 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:50.822510    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-lk7j8_40c7a4cf-1267-4e3d-ad93-c0d94078b181/kube-rbac-proxy-main/0.log"
Jul 22 12:48:50.846975 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:50.846948    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-lk7j8_40c7a4cf-1267-4e3d-ad93-c0d94078b181/kube-rbac-proxy-self/0.log"
Jul 22 12:48:51.045312 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.045179    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4kb42_1a08d3e8-2aac-4b4a-9e0f-8a19917d765c/node-exporter/0.log"
Jul 22 12:48:51.068798 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.068769    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4kb42_1a08d3e8-2aac-4b4a-9e0f-8a19917d765c/kube-rbac-proxy/0.log"
Jul 22 12:48:51.096489 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.096446    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4kb42_1a08d3e8-2aac-4b4a-9e0f-8a19917d765c/init-textfile/0.log"
Jul 22 12:48:51.299700 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.299621    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d6426c8b-4584-417e-9db0-b7aa3d962374/prometheus/0.log"
Jul 22 12:48:51.331439 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.331407    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d6426c8b-4584-417e-9db0-b7aa3d962374/config-reloader/0.log"
Jul 22 12:48:51.375297 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.375262    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d6426c8b-4584-417e-9db0-b7aa3d962374/thanos-sidecar/0.log"
Jul 22 12:48:51.413215 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.413188    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d6426c8b-4584-417e-9db0-b7aa3d962374/kube-rbac-proxy-web/0.log"
Jul 22 12:48:51.443884 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.443862    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d6426c8b-4584-417e-9db0-b7aa3d962374/kube-rbac-proxy/0.log"
Jul 22 12:48:51.469957 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.469927    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d6426c8b-4584-417e-9db0-b7aa3d962374/kube-rbac-proxy-thanos/0.log"
Jul 22 12:48:51.507626 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.507575    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d6426c8b-4584-417e-9db0-b7aa3d962374/init-config-reloader/0.log"
Jul 22 12:48:51.541339 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.541303    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-bwz5m_b2451b57-6b8f-4135-a644-24362c1344f0/prometheus-operator/0.log"
Jul 22 12:48:51.562587 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:51.562515    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-bwz5m_b2451b57-6b8f-4135-a644-24362c1344f0/kube-rbac-proxy/0.log"
Jul 22 12:48:52.804028 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.803986    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"]
Jul 22 12:48:52.810043 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.810010    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.821027 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.820999    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"]
Jul 22 12:48:52.826598 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.826573    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnqhw\" (UniqueName: \"kubernetes.io/projected/58f483f3-2df8-4275-8c37-263331b73d72-kube-api-access-cnqhw\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.826709 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.826643    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-proc\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.826778 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.826736    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-podres\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.826847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.826802    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-sys\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.826847 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.826833    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-lib-modules\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.928232 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.928187    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-podres\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.928585 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.928356    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-podres\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.928710 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.928569    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-sys\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.928710 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.928637    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-sys\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.928710 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.928663    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-lib-modules\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.928895 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.928772    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqhw\" (UniqueName: \"kubernetes.io/projected/58f483f3-2df8-4275-8c37-263331b73d72-kube-api-access-cnqhw\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.928895 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.928804    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-proc\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.928895 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.928869    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-lib-modules\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.929061 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.928908    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58f483f3-2df8-4275-8c37-263331b73d72-proc\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:52.939342 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:52.939315    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnqhw\" (UniqueName: \"kubernetes.io/projected/58f483f3-2df8-4275-8c37-263331b73d72-kube-api-access-cnqhw\") pod \"perf-node-gather-daemonset-z9bv6\" (UID: \"58f483f3-2df8-4275-8c37-263331b73d72\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:53.010773 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:53.010737    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-q8cs4_ea401827-12e1-4f67-8aa2-8151a4db724c/networking-console-plugin/0.log"
Jul 22 12:48:53.127129 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:53.127080    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:53.304499 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:53.302811    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"]
Jul 22 12:48:53.645657 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:53.645574    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6" event={"ID":"58f483f3-2df8-4275-8c37-263331b73d72","Type":"ContainerStarted","Data":"773a1b1a50c97765fddf3ba5ece8fd1e2fcb7218175ef37727e37a2a0504db0a"}
Jul 22 12:48:53.645657 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:53.645627    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6" event={"ID":"58f483f3-2df8-4275-8c37-263331b73d72","Type":"ContainerStarted","Data":"307dd00adc11362fa9b9ae92ecccd566093e0baffc5a1b302ca247de71784b30"}
Jul 22 12:48:53.645866 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:53.645708    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:53.666089 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:53.666036    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6" podStartSLOduration=1.666020828 podStartE2EDuration="1.666020828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:48:53.663724588 +0000 UTC m=+2822.685746289" watchObservedRunningTime="2026-07-22 12:48:53.666020828 +0000 UTC m=+2822.688042529"
Jul 22 12:48:55.952632 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:55.952600    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lfqfn_262cb757-0f60-4d3b-a354-db74abbd3176/dns/0.log"
Jul 22 12:48:55.982547 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:55.982518    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lfqfn_262cb757-0f60-4d3b-a354-db74abbd3176/kube-rbac-proxy/0.log"
Jul 22 12:48:56.113947 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:56.113921    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-shlxl_1dfea4d2-cebc-4c94-b4ac-3641e198d0e9/dns-node-resolver/0.log"
Jul 22 12:48:56.734351 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:56.734310    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-78446dd59d-8z6fg_13afccc2-342e-4e10-a82f-883dc273d9d6/registry/0.log"
Jul 22 12:48:56.785164 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:56.785139    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7pdv9_eef02bfb-180d-49b0-a4c6-52cf46557c5c/node-ca/0.log"
Jul 22 12:48:58.153656 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:58.153628    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fhvks_e5c60d29-9b5e-4b24-a0ac-4e8d118c2270/istio-proxy/0.log"
Jul 22 12:48:58.191934 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:58.191901    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fhvks_e5c60d29-9b5e-4b24-a0ac-4e8d118c2270/istio-proxy/1.log"
Jul 22 12:48:58.283718 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:58.283695    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d58b7fcb-9b8kf_62903732-d37c-4b43-b094-963c0910ac8b/router/0.log"
Jul 22 12:48:59.047833 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:59.047806    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wjj7v_5745c4de-e46c-4419-a288-70cf27b148a6/serve-healthcheck-canary/0.log"
Jul 22 12:48:59.638537 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:59.638507    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hq468_01d69365-7cc5-44c0-b707-0f5c2d343d73/kube-rbac-proxy/0.log"
Jul 22 12:48:59.664184 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:59.664156    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-z9bv6"
Jul 22 12:48:59.664443 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:59.664425    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hq468_01d69365-7cc5-44c0-b707-0f5c2d343d73/exporter/0.log"
Jul 22 12:48:59.696567 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:48:59.696541    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hq468_01d69365-7cc5-44c0-b707-0f5c2d343d73/extractor/0.log"
Jul 22 12:49:02.967094 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:49:02.967054    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58d867cfcf-zxv9g_8b94f66f-f022-414b-8784-603142645fde/manager/0.log"
Jul 22 12:49:03.188753 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:49:03.188667    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-979495d87-d4lcn_96084ba3-4eb5-4380-ba10-6e2bffc98431/manager/0.log"
Jul 22 12:49:03.213804 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:49:03.213774    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-979495d87-d4lcn_96084ba3-4eb5-4380-ba10-6e2bffc98431/copy-manifests/0.log"
Jul 22 12:49:03.867857 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:49:03.867802    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fhvks_e5c60d29-9b5e-4b24-a0ac-4e8d118c2270/istio-proxy/0.log"
Jul 22 12:49:03.905914 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:49:03.905885    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fhvks_e5c60d29-9b5e-4b24-a0ac-4e8d118c2270/istio-proxy/1.log"
Jul 22 12:49:03.962079 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:49:03.962050    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d58b7fcb-9b8kf_62903732-d37c-4b43-b094-963c0910ac8b/router/0.log"
Jul 22 12:49:04.536344 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:49:04.536297    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5fc976ff6d-q4gfr_9f631573-3147-442a-b1f9-c143db8db5a5/manager/0.log"
Jul 22 12:49:05.397497 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:49:05.397469    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fhvks_e5c60d29-9b5e-4b24-a0ac-4e8d118c2270/istio-proxy/0.log"
Jul 22 12:49:05.431700 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:49:05.431668    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fhvks_e5c60d29-9b5e-4b24-a0ac-4e8d118c2270/istio-proxy/1.log"
Jul 22 12:49:05.499345 ip-10-0-143-107 kubenswrapper[2584]: I0722 12:49:05.499312    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d58b7fcb-9b8kf_62903732-d37c-4b43-b094-963c0910ac8b/router/0.log"